2024-09-19T05:59:10.919 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-19T05:59:10.936 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T05:59:11.263 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911719 branch: squid description: fs/cephadm/multivolume/{0-start 1-mount 2-workload/dbench conf/{client mds mgr mon osd} distro/single-container-host overrides/{ignorelist_health pg_health}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7911719' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: debug ms: 1 debug osd: 20 osd op complaint time: 180 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 workunit: branch: squid sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - host.a - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - host.b - osd.4 - osd.5 - osd.6 - osd.7 - client.1 seed: 1747 sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 sleep_before_teardown: 0 subset: 176/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e targets: smithi053.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBW1y9QmgSDnq+alzqQOvTtHNVvJpuTerJguF/7zU6Orf2oxepDQXXp3ywhNwZTDlHNNwsDcYWRF1/9ZuG2wpOQ= smithi155.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMvFwg0VkytFX9PuTtYnM6ALsivYnzjYy/Mmve783p0jrDJvhXFrLGZwI+Yt0Zzg9TacHARiAEjJvzsMLVKfOPs= tasks: - install: null - cephadm: roleless: true - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create foo - ceph fs volume create bar - fs.ready: timeout: 300 - ceph-fuse: client.0: cephfs_name: foo - ceph-fuse: client.1: cephfs_name: bar - workunit: clients: all: - suites/dbench.sh - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install nvmetcli nvme-cli -y teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-18_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-19T05:59:11.263 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa; will attempt to use it 2024-09-19T05:59:11.264 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks 2024-09-19T05:59:11.264 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-19T05:59:11.266 INFO:teuthology.task.internal:Checking packages... 2024-09-19T05:59:11.287 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '2f599a46ad10e37c929e5c43da2f6907ddcd1961' 2024-09-19T05:59:11.288 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-19T05:59:11.288 INFO:teuthology.packaging:ref: None 2024-09-19T05:59:11.288 INFO:teuthology.packaging:tag: None 2024-09-19T05:59:11.288 INFO:teuthology.packaging:branch: squid 2024-09-19T05:59:11.288 INFO:teuthology.packaging:sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T05:59:11.288 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-19T05:59:11.498 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-363.g6fad5dd7 2024-09-19T05:59:11.500 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-19T05:59:11.507 INFO:teuthology.task.internal:no buildpackages task found 2024-09-19T05:59:11.507 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-19T05:59:11.513 INFO:teuthology.task.internal:Saving configuration 2024-09-19T05:59:11.525 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-19T05:59:11.532 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-19T05:59:11.552 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911719', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 05:54:06.646063', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBW1y9QmgSDnq+alzqQOvTtHNVvJpuTerJguF/7zU6Orf2oxepDQXXp3ywhNwZTDlHNNwsDcYWRF1/9ZuG2wpOQ='} 2024-09-19T05:59:11.570 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911719', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 05:54:06.644642', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMvFwg0VkytFX9PuTtYnM6ALsivYnzjYy/Mmve783p0jrDJvhXFrLGZwI+Yt0Zzg9TacHARiAEjJvzsMLVKfOPs='} 2024-09-19T05:59:11.571 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-19T05:59:11.577 INFO:teuthology.task.internal:roles: ubuntu@smithi053.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-09-19T05:59:11.578 INFO:teuthology.task.internal:roles: ubuntu@smithi155.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2024-09-19T05:59:11.578 INFO:teuthology.run_tasks:Running task console_log... 2024-09-19T05:59:11.626 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f810c2a3be0>, signals=[15]) 2024-09-19T05:59:11.626 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-19T05:59:11.632 INFO:teuthology.task.internal:Opening connections... 2024-09-19T05:59:11.632 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi053.front.sepia.ceph.com 2024-09-19T05:59:11.634 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:59:11.711 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi155.front.sepia.ceph.com 2024-09-19T05:59:11.711 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:59:11.784 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-19T05:59:11.791 DEBUG:teuthology.orchestra.run.smithi053:> uname -m 2024-09-19T05:59:11.809 INFO:teuthology.orchestra.run.smithi053.stdout:x86_64 2024-09-19T05:59:11.809 DEBUG:teuthology.orchestra.run.smithi053:> cat /etc/os-release 2024-09-19T05:59:11.866 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="CentOS Stream" 2024-09-19T05:59:11.867 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="9" 2024-09-19T05:59:11.867 INFO:teuthology.orchestra.run.smithi053.stdout:ID="centos" 2024-09-19T05:59:11.867 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE="rhel fedora" 2024-09-19T05:59:11.867 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="9" 2024-09-19T05:59:11.867 INFO:teuthology.orchestra.run.smithi053.stdout:PLATFORM_ID="platform:el9" 2024-09-19T05:59:11.867 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T05:59:11.867 INFO:teuthology.orchestra.run.smithi053.stdout:ANSI_COLOR="0;31" 2024-09-19T05:59:11.867 INFO:teuthology.orchestra.run.smithi053.stdout:LOGO="fedora-logo-icon" 2024-09-19T05:59:11.867 INFO:teuthology.orchestra.run.smithi053.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T05:59:11.867 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://centos.org/" 2024-09-19T05:59:11.867 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T05:59:11.868 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T05:59:11.868 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T05:59:11.868 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2024-09-19T05:59:11.891 DEBUG:teuthology.orchestra.run.smithi155:> uname -m 2024-09-19T05:59:11.910 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64 2024-09-19T05:59:11.910 DEBUG:teuthology.orchestra.run.smithi155:> cat /etc/os-release 2024-09-19T05:59:11.968 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="CentOS Stream" 2024-09-19T05:59:11.968 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="9" 2024-09-19T05:59:11.968 INFO:teuthology.orchestra.run.smithi155.stdout:ID="centos" 2024-09-19T05:59:11.968 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE="rhel fedora" 2024-09-19T05:59:11.968 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="9" 2024-09-19T05:59:11.969 INFO:teuthology.orchestra.run.smithi155.stdout:PLATFORM_ID="platform:el9" 2024-09-19T05:59:11.969 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T05:59:11.969 INFO:teuthology.orchestra.run.smithi155.stdout:ANSI_COLOR="0;31" 2024-09-19T05:59:11.969 INFO:teuthology.orchestra.run.smithi155.stdout:LOGO="fedora-logo-icon" 2024-09-19T05:59:11.969 INFO:teuthology.orchestra.run.smithi155.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T05:59:11.969 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://centos.org/" 2024-09-19T05:59:11.969 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T05:59:11.969 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T05:59:11.969 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T05:59:11.969 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2024-09-19T05:59:11.991 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-19T05:59:12.002 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-19T05:59:12.007 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-19T05:59:12.007 DEBUG:teuthology.orchestra.run.smithi053:> test '!' -e /home/ubuntu/cephtest 2024-09-19T05:59:12.010 DEBUG:teuthology.orchestra.run.smithi155:> test '!' -e /home/ubuntu/cephtest 2024-09-19T05:59:12.027 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-19T05:59:12.034 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-19T05:59:12.034 DEBUG:teuthology.orchestra.run.smithi053:> test -z $(ls -A /var/lib/ceph) 2024-09-19T05:59:12.067 DEBUG:teuthology.orchestra.run.smithi155:> test -z $(ls -A /var/lib/ceph) 2024-09-19T05:59:12.173 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-19T05:59:12.220 INFO:teuthology.run_tasks:Running task kernel... 2024-09-19T05:59:12.234 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-19T05:59:12.234 INFO:teuthology.task.kernel:config {'host.a': {'kdb': 1, 'sha1': 'distro'}, 'host.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-19T05:59:12.234 DEBUG:teuthology.orchestra.run.smithi053:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T05:59:12.235 DEBUG:teuthology.orchestra.run.smithi155:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T05:59:12.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:59:12.252 DEBUG:teuthology.orchestra.run.smithi053:> uname -r 2024-09-19T05:59:12.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:59:12.253 DEBUG:teuthology.orchestra.run.smithi155:> uname -r 2024-09-19T05:59:12.310 INFO:teuthology.orchestra.run.smithi053.stdout:5.14.0-503.el9.x86_64 2024-09-19T05:59:12.310 INFO:teuthology.task.kernel:Running kernel on smithi053: 5.14.0-503.el9.x86_64 2024-09-19T05:59:12.310 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum install -y kernel 2024-09-19T05:59:12.311 INFO:teuthology.orchestra.run.smithi155.stdout:5.14.0-503.el9.x86_64 2024-09-19T05:59:12.311 INFO:teuthology.task.kernel:Running kernel on smithi155: 5.14.0-503.el9.x86_64 2024-09-19T05:59:12.311 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum install -y kernel 2024-09-19T05:59:14.877 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.3 MB 00:00 2024-09-19T05:59:15.786 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.3 MB 00:01 2024-09-19T05:59:16.786 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - AppStream 24 MB/s | 20 MB 00:00 2024-09-19T05:59:21.179 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-19T05:59:21.437 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - AppStream 4.5 MB/s | 20 MB 00:04 2024-09-19T05:59:22.929 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 19 kB 00:00 2024-09-19T05:59:23.767 INFO:teuthology.orchestra.run.smithi155.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-19T05:59:26.629 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - CRB 5.1 MB/s | 6.5 MB 00:01 2024-09-19T05:59:28.996 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - Extras packages 17 kB/s | 19 kB 00:01 2024-09-19T05:59:29.189 INFO:teuthology.orchestra.run.smithi155.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-09-19T05:59:29.809 INFO:teuthology.orchestra.run.smithi053.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-19T05:59:31.170 INFO:teuthology.orchestra.run.smithi155.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-19T05:59:31.171 INFO:teuthology.orchestra.run.smithi155.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-19T05:59:31.255 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-09-19T05:59:31.263 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T05:59:31.263 INFO:teuthology.orchestra.run.smithi155.stdout: Package Architecture Version Repository Size 2024-09-19T05:59:31.263 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T05:59:31.263 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-09-19T05:59:31.263 INFO:teuthology.orchestra.run.smithi155.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-19T05:59:31.263 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-09-19T05:59:31.263 INFO:teuthology.orchestra.run.smithi155.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-19T05:59:31.263 INFO:teuthology.orchestra.run.smithi155.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-19T05:59:31.263 INFO:teuthology.orchestra.run.smithi155.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-19T05:59:31.263 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T05:59:31.263 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-09-19T05:59:31.264 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T05:59:31.264 INFO:teuthology.orchestra.run.smithi155.stdout:Install 4 Packages 2024-09-19T05:59:31.264 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T05:59:31.264 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 79 M 2024-09-19T05:59:31.264 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 126 M 2024-09-19T05:59:31.264 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-09-19T05:59:31.407 INFO:teuthology.orchestra.run.smithi155.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 376 kB/s | 36 kB 00:00 2024-09-19T05:59:31.824 INFO:teuthology.orchestra.run.smithi155.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-09-19T05:59:32.209 INFO:teuthology.orchestra.run.smithi155.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 36 MB/s | 29 MB 00:00 2024-09-19T05:59:32.525 INFO:teuthology.orchestra.run.smithi155.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 28 MB/s | 35 MB 00:01 2024-09-19T05:59:32.525 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-09-19T05:59:32.525 INFO:teuthology.orchestra.run.smithi155.stdout:Total 62 MB/s | 79 MB 00:01 2024-09-19T05:59:33.211 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-09-19T05:59:33.579 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-09-19T05:59:33.579 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-09-19T05:59:34.561 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-09-19T05:59:34.562 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-09-19T05:59:35.531 INFO:teuthology.orchestra.run.smithi053.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-09-19T05:59:36.998 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-09-19T05:59:37.463 INFO:teuthology.orchestra.run.smithi053.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-19T05:59:37.463 INFO:teuthology.orchestra.run.smithi053.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-19T05:59:37.563 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-19T05:59:37.568 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T05:59:37.568 INFO:teuthology.orchestra.run.smithi053.stdout: Package Architecture Version Repository Size 2024-09-19T05:59:37.568 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T05:59:37.568 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-19T05:59:37.568 INFO:teuthology.orchestra.run.smithi053.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-19T05:59:37.568 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-19T05:59:37.568 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-19T05:59:37.568 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-19T05:59:37.569 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-19T05:59:37.569 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T05:59:37.569 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-19T05:59:37.569 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T05:59:37.569 INFO:teuthology.orchestra.run.smithi053.stdout:Install 4 Packages 2024-09-19T05:59:37.569 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T05:59:37.569 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 79 M 2024-09-19T05:59:37.570 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 126 M 2024-09-19T05:59:37.570 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-19T05:59:37.641 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-19T05:59:37.700 INFO:teuthology.orchestra.run.smithi053.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 427 kB/s | 36 kB 00:00 2024-09-19T05:59:37.732 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T05:59:38.283 INFO:teuthology.orchestra.run.smithi053.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-09-19T05:59:38.725 INFO:teuthology.orchestra.run.smithi053.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 31 MB/s | 35 MB 00:01 2024-09-19T05:59:39.175 INFO:teuthology.orchestra.run.smithi053.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 19 MB/s | 29 MB 00:01 2024-09-19T05:59:39.175 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-19T05:59:39.175 INFO:teuthology.orchestra.run.smithi053.stdout:Total 49 MB/s | 79 MB 00:01 2024-09-19T05:59:39.888 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-19T05:59:40.295 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-19T05:59:40.295 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-19T05:59:40.517 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T05:59:40.643 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T05:59:41.261 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-19T05:59:41.262 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-19T05:59:43.831 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-19T05:59:44.488 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-19T05:59:44.856 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T05:59:45.472 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T05:59:45.593 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:59:47.384 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T05:59:47.522 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T05:59:50.428 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:59:52.648 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T05:59:52.698 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:59:57.836 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T06:00:28.976 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T06:00:29.015 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-19T06:00:29.380 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T06:00:29.380 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-19T06:00:29.380 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T06:00:29.380 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T06:00:30.221 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T06:00:30.222 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:00:30.222 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-09-19T06:00:30.222 INFO:teuthology.orchestra.run.smithi155.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-19T06:00:30.222 INFO:teuthology.orchestra.run.smithi155.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-19T06:00:30.222 INFO:teuthology.orchestra.run.smithi155.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-19T06:00:30.222 INFO:teuthology.orchestra.run.smithi155.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-19T06:00:30.222 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:00:30.222 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-09-19T06:00:30.576 DEBUG:teuthology.orchestra.run.smithi155:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T06:00:30.659 INFO:teuthology.orchestra.run.smithi155.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T06:00:30.659 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T06:00:30.659 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-19T06:00:30.659 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T06:00:30.659 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T06:00:30.659 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-09-19T06:00:30.660 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi155.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T06:00:30.660 DEBUG:teuthology.orchestra.run.smithi155:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T06:00:30.720 INFO:teuthology.orchestra.run.smithi155.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T06:00:30.720 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T06:00:30.720 DEBUG:teuthology.orchestra.run.smithi155:> sudo rpm -qi grub2-tools 2024-09-19T06:00:30.752 INFO:teuthology.orchestra.run.smithi155.stdout:Name : grub2-tools 2024-09-19T06:00:30.752 INFO:teuthology.orchestra.run.smithi155.stdout:Epoch : 1 2024-09-19T06:00:30.752 INFO:teuthology.orchestra.run.smithi155.stdout:Version : 2.06 2024-09-19T06:00:30.752 INFO:teuthology.orchestra.run.smithi155.stdout:Release : 82.el9 2024-09-19T06:00:30.752 INFO:teuthology.orchestra.run.smithi155.stdout:Architecture: x86_64 2024-09-19T06:00:30.752 INFO:teuthology.orchestra.run.smithi155.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-19T06:00:30.753 INFO:teuthology.orchestra.run.smithi155.stdout:Group : Unspecified 2024-09-19T06:00:30.753 INFO:teuthology.orchestra.run.smithi155.stdout:Size : 8274670 2024-09-19T06:00:30.753 INFO:teuthology.orchestra.run.smithi155.stdout:License : GPLv3+ 2024-09-19T06:00:30.753 INFO:teuthology.orchestra.run.smithi155.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-19T06:00:30.753 INFO:teuthology.orchestra.run.smithi155.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-19T06:00:30.753 INFO:teuthology.orchestra.run.smithi155.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-19T06:00:30.753 INFO:teuthology.orchestra.run.smithi155.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-19T06:00:30.753 INFO:teuthology.orchestra.run.smithi155.stdout:Packager : builder@centos.org 2024-09-19T06:00:30.753 INFO:teuthology.orchestra.run.smithi155.stdout:Vendor : CentOS 2024-09-19T06:00:30.753 INFO:teuthology.orchestra.run.smithi155.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-19T06:00:30.753 INFO:teuthology.orchestra.run.smithi155.stdout:Summary : Support tools for GRUB. 2024-09-19T06:00:30.753 INFO:teuthology.orchestra.run.smithi155.stdout:Description : 2024-09-19T06:00:30.753 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:00:30.753 INFO:teuthology.orchestra.run.smithi155.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-19T06:00:30.754 INFO:teuthology.orchestra.run.smithi155.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-19T06:00:30.754 INFO:teuthology.orchestra.run.smithi155.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-19T06:00:30.754 INFO:teuthology.orchestra.run.smithi155.stdout:hardware devices. 2024-09-19T06:00:30.754 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:00:30.754 INFO:teuthology.orchestra.run.smithi155.stdout:This subpackage provides tools for support of all platforms. 2024-09-19T06:00:30.755 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-19T06:00:30.755 INFO:teuthology.task.kernel:Updating grub on smithi155 to boot 5.14.0-508.el9.x86_64 2024-09-19T06:00:30.755 DEBUG:teuthology.orchestra.run.smithi155:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-19T06:00:31.383 INFO:teuthology.orchestra.run.smithi155.stderr:Generating grub configuration file ... 2024-09-19T06:00:32.759 INFO:teuthology.orchestra.run.smithi155.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-19T06:00:32.805 INFO:teuthology.orchestra.run.smithi155.stderr:done 2024-09-19T06:00:32.807 DEBUG:teuthology.orchestra.run.smithi155:> mktemp 2024-09-19T06:00:32.822 INFO:teuthology.orchestra.run.smithi155.stdout:/tmp/tmp.Na6nHcIq9w 2024-09-19T06:00:32.822 DEBUG:teuthology.orchestra.run.smithi155:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Na6nHcIq9w 2024-09-19T06:00:32.892 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0666 /tmp/tmp.Na6nHcIq9w 2024-09-19T06:00:33.039 DEBUG:teuthology.orchestra.remote:smithi155:/tmp/tmp.Na6nHcIq9w is 6KB 2024-09-19T06:00:33.088 DEBUG:teuthology.orchestra.run.smithi155:> rm -fr /tmp/tmp.Na6nHcIq9w 2024-09-19T06:00:33.103 DEBUG:teuthology.orchestra.run.smithi155:> sudo /bin/ls /boot/loader/entries || true 2024-09-19T06:00:33.167 INFO:teuthology.orchestra.run.smithi155.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-19T06:00:33.167 INFO:teuthology.orchestra.run.smithi155.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-19T06:00:33.167 INFO:teuthology.orchestra.run.smithi155.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-19T06:00:33.168 INFO:teuthology.orchestra.run.smithi155.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-19T06:00:33.169 DEBUG:teuthology.orchestra.run.smithi155:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-19T06:00:33.334 DEBUG:teuthology.orchestra.run.smithi155:> sudo shutdown -r now 2024-09-19T06:00:37.195 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T06:00:37.236 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-19T06:00:37.590 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T06:00:37.590 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-19T06:00:37.590 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T06:00:37.590 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T06:00:38.535 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T06:00:38.535 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:00:38.535 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-19T06:00:38.536 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-19T06:00:38.536 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-19T06:00:38.536 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-19T06:00:38.536 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-19T06:00:38.536 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:00:38.536 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-19T06:00:38.895 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T06:00:38.985 INFO:teuthology.orchestra.run.smithi053.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T06:00:38.985 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T06:00:38.985 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-19T06:00:38.985 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T06:00:38.985 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T06:00:38.985 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-09-19T06:00:38.985 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi053.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T06:00:38.986 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T06:00:39.044 INFO:teuthology.orchestra.run.smithi053.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T06:00:39.044 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T06:00:39.044 DEBUG:teuthology.orchestra.run.smithi053:> sudo rpm -qi grub2-tools 2024-09-19T06:00:39.079 INFO:teuthology.orchestra.run.smithi053.stdout:Name : grub2-tools 2024-09-19T06:00:39.079 INFO:teuthology.orchestra.run.smithi053.stdout:Epoch : 1 2024-09-19T06:00:39.079 INFO:teuthology.orchestra.run.smithi053.stdout:Version : 2.06 2024-09-19T06:00:39.079 INFO:teuthology.orchestra.run.smithi053.stdout:Release : 82.el9 2024-09-19T06:00:39.079 INFO:teuthology.orchestra.run.smithi053.stdout:Architecture: x86_64 2024-09-19T06:00:39.079 INFO:teuthology.orchestra.run.smithi053.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-19T06:00:39.080 INFO:teuthology.orchestra.run.smithi053.stdout:Group : Unspecified 2024-09-19T06:00:39.080 INFO:teuthology.orchestra.run.smithi053.stdout:Size : 8274670 2024-09-19T06:00:39.080 INFO:teuthology.orchestra.run.smithi053.stdout:License : GPLv3+ 2024-09-19T06:00:39.080 INFO:teuthology.orchestra.run.smithi053.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-19T06:00:39.080 INFO:teuthology.orchestra.run.smithi053.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-19T06:00:39.080 INFO:teuthology.orchestra.run.smithi053.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-19T06:00:39.080 INFO:teuthology.orchestra.run.smithi053.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-19T06:00:39.080 INFO:teuthology.orchestra.run.smithi053.stdout:Packager : builder@centos.org 2024-09-19T06:00:39.080 INFO:teuthology.orchestra.run.smithi053.stdout:Vendor : CentOS 2024-09-19T06:00:39.080 INFO:teuthology.orchestra.run.smithi053.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-19T06:00:39.080 INFO:teuthology.orchestra.run.smithi053.stdout:Summary : Support tools for GRUB. 2024-09-19T06:00:39.080 INFO:teuthology.orchestra.run.smithi053.stdout:Description : 2024-09-19T06:00:39.080 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:00:39.081 INFO:teuthology.orchestra.run.smithi053.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-19T06:00:39.081 INFO:teuthology.orchestra.run.smithi053.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-19T06:00:39.081 INFO:teuthology.orchestra.run.smithi053.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-19T06:00:39.081 INFO:teuthology.orchestra.run.smithi053.stdout:hardware devices. 2024-09-19T06:00:39.081 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:00:39.081 INFO:teuthology.orchestra.run.smithi053.stdout:This subpackage provides tools for support of all platforms. 2024-09-19T06:00:39.082 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-19T06:00:39.082 INFO:teuthology.task.kernel:Updating grub on smithi053 to boot 5.14.0-508.el9.x86_64 2024-09-19T06:00:39.082 DEBUG:teuthology.orchestra.run.smithi053:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-19T06:00:39.810 INFO:teuthology.orchestra.run.smithi053.stderr:Generating grub configuration file ... 2024-09-19T06:00:41.162 INFO:teuthology.orchestra.run.smithi053.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-19T06:00:41.183 INFO:teuthology.orchestra.run.smithi053.stderr:done 2024-09-19T06:00:41.185 DEBUG:teuthology.orchestra.run.smithi053:> mktemp 2024-09-19T06:00:41.200 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/tmp.wd9dqYKNZ1 2024-09-19T06:00:41.201 DEBUG:teuthology.orchestra.run.smithi053:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.wd9dqYKNZ1 2024-09-19T06:00:41.267 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0666 /tmp/tmp.wd9dqYKNZ1 2024-09-19T06:00:41.402 DEBUG:teuthology.orchestra.remote:smithi053:/tmp/tmp.wd9dqYKNZ1 is 6KB 2024-09-19T06:00:41.451 DEBUG:teuthology.orchestra.run.smithi053:> rm -fr /tmp/tmp.wd9dqYKNZ1 2024-09-19T06:00:41.465 DEBUG:teuthology.orchestra.run.smithi053:> sudo /bin/ls /boot/loader/entries || true 2024-09-19T06:00:41.527 INFO:teuthology.orchestra.run.smithi053.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-19T06:00:41.527 INFO:teuthology.orchestra.run.smithi053.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-19T06:00:41.528 INFO:teuthology.orchestra.run.smithi053.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-19T06:00:41.528 INFO:teuthology.orchestra.run.smithi053.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-19T06:00:41.529 DEBUG:teuthology.orchestra.run.smithi053:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-19T06:00:41.676 DEBUG:teuthology.orchestra.run.smithi053:> sudo shutdown -r now 2024-09-19T06:01:03.358 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-09-19T06:01:03.359 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-09-19T06:01:03.360 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T06:01:11.680 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-09-19T06:01:11.681 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-19T06:01:11.682 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T06:01:21.725 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-09-19T06:01:30.173 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.53 2024-09-19T06:01:30.727 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-09-19T06:01:30.728 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T06:01:33.789 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-09-19T06:01:39.176 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-19T06:01:39.177 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T06:01:45.796 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-09-19T06:01:45.797 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T06:01:57.568 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.53 2024-09-19T06:02:04.225 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-09-19T06:02:09.573 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-19T06:02:09.574 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T06:02:10.004 DEBUG:teuthology.orchestra.run.smithi053:> true 2024-09-19T06:02:10.402 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-19T06:02:10.403 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-508.el9.x86_64"... 2024-09-19T06:02:10.403 DEBUG:teuthology.orchestra.run.smithi053:> uname -r 2024-09-19T06:02:10.419 INFO:teuthology.orchestra.run.smithi053.stdout:5.14.0-508.el9.x86_64 2024-09-19T06:02:10.419 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-19T06:02:10.419 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T06:02:10.419 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-19T06:02:11.420 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-09-19T06:02:11.420 DEBUG:teuthology.orchestra.run.smithi053:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-19T06:02:11.520 INFO:teuthology.orchestra.run.smithi053.stdout:ttyS1 2024-09-19T06:02:11.543 DEBUG:teuthology.parallel:result is None 2024-09-19T06:02:19.226 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-09-19T06:02:19.227 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T06:02:19.603 DEBUG:teuthology.orchestra.run.smithi155:> true 2024-09-19T06:02:19.989 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-09-19T06:02:19.989 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-508.el9.x86_64"... 2024-09-19T06:02:19.989 DEBUG:teuthology.orchestra.run.smithi155:> uname -r 2024-09-19T06:02:20.003 INFO:teuthology.orchestra.run.smithi155.stdout:5.14.0-508.el9.x86_64 2024-09-19T06:02:20.003 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-19T06:02:20.003 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T06:02:20.004 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-19T06:02:21.004 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-09-19T06:02:21.005 DEBUG:teuthology.orchestra.run.smithi155:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-19T06:02:21.134 INFO:teuthology.orchestra.run.smithi155.stdout:ttyS1 2024-09-19T06:02:21.168 DEBUG:teuthology.parallel:result is None 2024-09-19T06:02:21.168 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-19T06:02:21.178 INFO:teuthology.task.internal:Creating test directory... 2024-09-19T06:02:21.178 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T06:02:21.182 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T06:02:21.200 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-19T06:02:21.206 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-19T06:02:21.323 INFO:teuthology.task.internal:Creating archive directory... 2024-09-19T06:02:21.324 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T06:02:21.327 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T06:02:21.367 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-19T06:02:21.393 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-19T06:02:21.394 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T06:02:21.397 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T06:02:21.426 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T06:02:21.432 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T06:02:21.438 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T06:02:21.442 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T06:02:21.443 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-19T06:02:21.451 INFO:teuthology.task.internal:Configuring sudo... 2024-09-19T06:02:21.451 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T06:02:21.482 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T06:02:21.511 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-19T06:02:21.521 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-19T06:02:21.521 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T06:02:21.552 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T06:02:21.569 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T06:02:21.652 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T06:02:21.702 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T06:02:21.787 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T06:02:21.828 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-19T06:02:21.828 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T06:02:21.895 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T06:02:21.936 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T06:02:21.987 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T06:02:22.089 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T06:02:22.156 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-19T06:02:22.156 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T06:02:22.262 DEBUG:teuthology.orchestra.run.smithi053:> sudo service rsyslog restart 2024-09-19T06:02:22.265 DEBUG:teuthology.orchestra.run.smithi155:> sudo service rsyslog restart 2024-09-19T06:02:22.319 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T06:02:22.403 INFO:teuthology.orchestra.run.smithi155.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T06:02:22.794 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-19T06:02:22.801 INFO:teuthology.task.internal:Starting timer... 2024-09-19T06:02:22.801 INFO:teuthology.run_tasks:Running task pcp... 2024-09-19T06:02:22.813 INFO:teuthology.run_tasks:Running task selinux... 2024-09-19T06:02:22.822 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-19T06:02:22.822 DEBUG:teuthology.orchestra.run.smithi053:> sudo service auditd rotate 2024-09-19T06:02:22.888 INFO:teuthology.orchestra.run.smithi053.stdout:Rotating logs: 2024-09-19T06:02:22.890 DEBUG:teuthology.orchestra.run.smithi155:> sudo service auditd rotate 2024-09-19T06:02:23.005 INFO:teuthology.orchestra.run.smithi155.stdout:Rotating logs: 2024-09-19T06:02:23.009 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-19T06:02:23.009 DEBUG:teuthology.orchestra.run.smithi053:> /usr/sbin/getenforce 2024-09-19T06:02:23.037 INFO:teuthology.orchestra.run.smithi053.stdout:Permissive 2024-09-19T06:02:23.038 DEBUG:teuthology.orchestra.run.smithi155:> /usr/sbin/getenforce 2024-09-19T06:02:23.058 INFO:teuthology.orchestra.run.smithi155.stdout:Permissive 2024-09-19T06:02:23.058 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi053.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi155.front.sepia.ceph.com': 'permissive'} 2024-09-19T06:02:23.059 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T06:02:23.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T06:02:23.105 DEBUG:teuthology.orchestra.run.smithi155:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T06:02:23.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T06:02:23.131 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-19T06:02:23.132 DEBUG:teuthology.orchestra.run.smithi053:> sudo /usr/sbin/setenforce permissive 2024-09-19T06:02:23.181 DEBUG:teuthology.orchestra.run.smithi155:> sudo /usr/sbin/setenforce permissive 2024-09-19T06:02:23.214 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-19T06:02:23.223 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-19T06:02:23.235 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-19T06:02:23.236 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi053.front.sepia.ceph.com,smithi155.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-19T06:06:10.343 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi053.front.sepia.ceph.com'), Remote(name='ubuntu@smithi155.front.sepia.ceph.com')] 2024-09-19T06:06:10.344 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-19T06:06:10.345 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T06:06:10.427 DEBUG:teuthology.orchestra.run.smithi053:> true 2024-09-19T06:06:10.510 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-19T06:06:10.511 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-09-19T06:06:10.512 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T06:06:10.590 DEBUG:teuthology.orchestra.run.smithi155:> true 2024-09-19T06:06:10.665 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-09-19T06:06:10.665 INFO:teuthology.run_tasks:Running task clock... 2024-09-19T06:06:10.676 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-19T06:06:10.676 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T06:06:10.677 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T06:06:10.679 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T06:06:10.679 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T06:06:10.709 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T06:06:10.725 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T06:06:10.736 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T06:06:10.751 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T06:06:10.752 INFO:teuthology.orchestra.run.smithi053.stderr:sudo: ntpd: command not found 2024-09-19T06:06:10.764 INFO:teuthology.orchestra.run.smithi053.stdout:506 Cannot talk to daemon 2024-09-19T06:06:10.777 INFO:teuthology.orchestra.run.smithi155.stderr:sudo: ntpd: command not found 2024-09-19T06:06:10.780 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T06:06:10.788 INFO:teuthology.orchestra.run.smithi155.stdout:506 Cannot talk to daemon 2024-09-19T06:06:10.795 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T06:06:10.802 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T06:06:10.817 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T06:06:10.858 INFO:teuthology.orchestra.run.smithi053.stderr:bash: line 1: ntpq: command not found 2024-09-19T06:06:10.862 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T06:06:10.863 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2024-09-19T06:06:10.863 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:06:10.863 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:06:10.863 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:06:10.863 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:06:10.874 INFO:teuthology.orchestra.run.smithi155.stderr:bash: line 1: ntpq: command not found 2024-09-19T06:06:10.878 INFO:teuthology.orchestra.run.smithi155.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T06:06:10.878 INFO:teuthology.orchestra.run.smithi155.stdout:=============================================================================== 2024-09-19T06:06:10.878 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:06:10.878 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:06:10.878 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:06:10.878 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:06:10.879 INFO:teuthology.run_tasks:Running task install... 2024-09-19T06:06:10.887 DEBUG:teuthology.task.install:project ceph 2024-09-19T06:06:10.887 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}} 2024-09-19T06:06:10.887 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'} 2024-09-19T06:06:10.888 INFO:teuthology.task.install:Using flavor: default 2024-09-19T06:06:10.893 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-19T06:06:10.893 INFO:teuthology.task.install:extra packages: [] 2024-09-19T06:06:10.894 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T06:06:10.894 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:06:10.896 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T06:06:10.896 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:06:11.053 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T06:06:11.053 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T06:06:11.062 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T06:06:11.063 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T06:06:11.169 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T06:06:11.169 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-19T06:06:11.169 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T06:06:11.180 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T06:06:11.180 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-19T06:06:11.180 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T06:06:11.197 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-19T06:06:11.197 DEBUG:teuthology.orchestra.run.smithi053:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T06:06:11.206 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-19T06:06:11.207 DEBUG:teuthology.orchestra.run.smithi155:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T06:06:11.264 DEBUG:teuthology.orchestra.run.smithi053:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T06:06:11.272 DEBUG:teuthology.orchestra.run.smithi155:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T06:06:11.341 DEBUG:teuthology.orchestra.run.smithi053:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T06:06:11.348 DEBUG:teuthology.orchestra.run.smithi155:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T06:06:11.406 INFO:teuthology.orchestra.run.smithi053.stdout:check_obsoletes = 1 2024-09-19T06:06:11.408 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum clean all 2024-09-19T06:06:11.412 INFO:teuthology.orchestra.run.smithi155.stdout:check_obsoletes = 1 2024-09-19T06:06:11.414 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum clean all 2024-09-19T06:06:11.644 INFO:teuthology.orchestra.run.smithi053.stdout:45 files removed 2024-09-19T06:06:11.657 INFO:teuthology.orchestra.run.smithi155.stdout:45 files removed 2024-09-19T06:06:11.671 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-radosgw 2024-09-19T06:06:11.684 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-radosgw 2024-09-19T06:06:12.252 INFO:teuthology.orchestra.run.smithi155.stdout:ceph packages for x86_64 294 kB/s | 84 kB 00:00 2024-09-19T06:06:12.285 INFO:teuthology.orchestra.run.smithi053.stdout:ceph packages for x86_64 265 kB/s | 84 kB 00:00 2024-09-19T06:06:12.538 INFO:teuthology.orchestra.run.smithi155.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-19T06:06:12.590 INFO:teuthology.orchestra.run.smithi053.stdout:ceph noarch packages 45 kB/s | 12 kB 00:00 2024-09-19T06:06:12.814 INFO:teuthology.orchestra.run.smithi155.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-19T06:06:12.862 INFO:teuthology.orchestra.run.smithi053.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-19T06:06:15.062 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - BaseOS 3.8 MB/s | 8.3 MB 00:02 2024-09-19T06:06:15.373 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - BaseOS 3.3 MB/s | 8.3 MB 00:02 2024-09-19T06:06:20.388 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - AppStream 4.9 MB/s | 20 MB 00:04 2024-09-19T06:06:24.323 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - AppStream 2.6 MB/s | 20 MB 00:07 2024-09-19T06:06:25.622 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - CRB 5.1 MB/s | 6.5 MB 00:01 2024-09-19T06:06:27.964 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 19 kB 00:01 2024-09-19T06:06:28.818 INFO:teuthology.orchestra.run.smithi053.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-19T06:06:30.215 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 6.5 MB 00:02 2024-09-19T06:06:34.691 INFO:teuthology.orchestra.run.smithi053.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-19T06:06:37.010 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-19T06:06:37.025 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:06:37.025 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-19T06:06:37.025 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:06:37.025 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-19T06:06:37.025 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T06:06:37.025 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrading: 2024-09-19T06:06:37.025 INFO:teuthology.orchestra.run.smithi053.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T06:06:37.026 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T06:06:37.026 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-19T06:06:37.026 INFO:teuthology.orchestra.run.smithi053.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T06:06:37.026 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T06:06:37.026 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T06:06:37.026 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T06:06:37.026 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T06:06:37.026 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T06:06:37.026 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T06:06:37.026 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T06:06:37.026 INFO:teuthology.orchestra.run.smithi053.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T06:06:37.026 INFO:teuthology.orchestra.run.smithi053.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T06:06:37.026 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T06:06:37.027 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T06:06:37.027 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T06:06:37.027 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T06:06:37.027 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T06:06:37.027 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T06:06:37.027 INFO:teuthology.orchestra.run.smithi053.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T06:06:37.027 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T06:06:37.027 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T06:06:37.027 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T06:06:37.027 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T06:06:37.027 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T06:06:37.027 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T06:06:37.027 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T06:06:37.028 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T06:06:37.028 INFO:teuthology.orchestra.run.smithi053.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T06:06:37.028 INFO:teuthology.orchestra.run.smithi053.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T06:06:37.028 INFO:teuthology.orchestra.run.smithi053.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T06:06:37.028 INFO:teuthology.orchestra.run.smithi053.stdout:Installing weak dependencies: 2024-09-19T06:06:37.028 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T06:06:37.028 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:06:37.028 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-19T06:06:37.028 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:06:37.028 INFO:teuthology.orchestra.run.smithi053.stdout:Install 30 Packages 2024-09-19T06:06:37.028 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrade 2 Packages 2024-09-19T06:06:37.028 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:06:37.028 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 60 M 2024-09-19T06:06:37.029 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-19T06:06:37.800 INFO:teuthology.orchestra.run.smithi053.stdout:(1/32): ceph-base-19.1.1-361.g2f599a46.el9.x86_ 11 MB/s | 5.4 MB 00:00 2024-09-19T06:06:37.867 INFO:teuthology.orchestra.run.smithi053.stdout:(2/32): ceph-selinux-19.1.1-361.g2f599a46.el9.x 378 kB/s | 25 kB 00:00 2024-09-19T06:06:37.967 INFO:teuthology.orchestra.run.smithi053.stdout:(3/32): libcephfs2-19.1.1-361.g2f599a46.el9.x86 7.3 MB/s | 751 kB 00:00 2024-09-19T06:06:38.093 INFO:teuthology.orchestra.run.smithi053.stdout:(4/32): ceph-radosgw-19.1.1-361.g2f599a46.el9.x 14 MB/s | 11 MB 00:00 2024-09-19T06:06:38.126 INFO:teuthology.orchestra.run.smithi053.stdout:(5/32): libradosstriper1-19.1.1-361.g2f599a46.e 3.1 MB/s | 506 kB 00:00 2024-09-19T06:06:38.193 INFO:teuthology.orchestra.run.smithi053.stdout:(6/32): python3-ceph-argparse-19.1.1-361.g2f599 677 kB/s | 45 kB 00:00 2024-09-19T06:06:38.328 INFO:teuthology.orchestra.run.smithi053.stdout:(7/32): librgw2-19.1.1-361.g2f599a46.el9.x86_64 23 MB/s | 5.3 MB 00:00 2024-09-19T06:06:38.352 INFO:teuthology.orchestra.run.smithi053.stdout:(8/32): python3-ceph-common-19.1.1-361.g2f599a4 859 kB/s | 136 kB 00:00 2024-09-19T06:06:38.377 INFO:teuthology.orchestra.run.smithi053.stdout:(9/32): python3-cephfs-19.1.1-361.g2f599a46.el9 3.1 MB/s | 161 kB 00:00 2024-09-19T06:06:38.411 INFO:teuthology.orchestra.run.smithi053.stdout:(10/32): python3-rados-19.1.1-361.g2f599a46.el9 5.3 MB/s | 320 kB 00:00 2024-09-19T06:06:38.436 INFO:teuthology.orchestra.run.smithi053.stdout:(11/32): python3-rbd-19.1.1-361.g2f599a46.el9.x 5.0 MB/s | 300 kB 00:00 2024-09-19T06:06:38.461 INFO:teuthology.orchestra.run.smithi053.stdout:(12/32): python3-rgw-19.1.1-361.g2f599a46.el9.x 1.9 MB/s | 99 kB 00:00 2024-09-19T06:06:38.720 INFO:teuthology.orchestra.run.smithi053.stdout:(13/32): ceph-common-19.1.1-361.g2f599a46.el9.x 15 MB/s | 21 MB 00:01 2024-09-19T06:06:38.770 INFO:teuthology.orchestra.run.smithi053.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 320 kB/s | 107 kB 00:00 2024-09-19T06:06:38.829 INFO:teuthology.orchestra.run.smithi053.stdout:(15/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 436 kB/s | 160 kB 00:00 2024-09-19T06:06:38.879 INFO:teuthology.orchestra.run.smithi053.stdout:(16/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 285 kB/s | 45 kB 00:00 2024-09-19T06:06:38.954 INFO:teuthology.orchestra.run.smithi053.stdout:(17/32): librdkafka-1.6.1-102.el9.x86_64.rpm 3.5 MB/s | 662 kB 00:00 2024-09-19T06:06:38.979 INFO:teuthology.orchestra.run.smithi053.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 421 kB/s | 42 kB 00:00 2024-09-19T06:06:39.005 INFO:teuthology.orchestra.run.smithi053.stdout:(19/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.6 MB/s | 292 kB 00:00 2024-09-19T06:06:39.030 INFO:teuthology.orchestra.run.smithi053.stdout:(20/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-09-19T06:06:39.055 INFO:teuthology.orchestra.run.smithi053.stdout:(21/32): qatlib-service-24.02.0-1.el9.x86_64.rp 475 kB/s | 36 kB 00:00 2024-09-19T06:06:39.081 INFO:teuthology.orchestra.run.smithi053.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 628 kB/s | 47 kB 00:00 2024-09-19T06:06:39.139 INFO:teuthology.orchestra.run.smithi053.stdout:(23/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.8 MB/s | 308 kB 00:00 2024-09-19T06:06:39.164 INFO:teuthology.orchestra.run.smithi053.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-19T06:06:39.190 INFO:teuthology.orchestra.run.smithi053.stdout:(25/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 280 kB/s | 26 kB 00:00 2024-09-19T06:06:39.215 INFO:teuthology.orchestra.run.smithi053.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-19T06:06:39.249 INFO:teuthology.orchestra.run.smithi053.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-19T06:06:39.257 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - Extras packages 2.4 kB/s | 19 kB 00:07 2024-09-19T06:06:39.290 INFO:teuthology.orchestra.run.smithi053.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 839 kB 00:00 2024-09-19T06:06:39.357 INFO:teuthology.orchestra.run.smithi053.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-19T06:06:39.441 INFO:teuthology.orchestra.run.smithi053.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-19T06:06:39.533 INFO:teuthology.orchestra.run.smithi053.stdout:(31/32): librados2-19.1.1-361.g2f599a46.el9.x86 14 MB/s | 3.4 MB 00:00 2024-09-19T06:06:39.592 INFO:teuthology.orchestra.run.smithi053.stdout:(32/32): librbd1-19.1.1-361.g2f599a46.el9.x86_6 14 MB/s | 3.2 MB 00:00 2024-09-19T06:06:39.597 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:06:39.597 INFO:teuthology.orchestra.run.smithi053.stdout:Total 23 MB/s | 60 MB 00:02 2024-09-19T06:06:39.704 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-19T06:06:39.744 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-19T06:06:39.744 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-19T06:06:40.128 INFO:teuthology.orchestra.run.smithi155.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-19T06:06:40.426 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-19T06:06:40.427 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-19T06:06:41.977 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-19T06:06:42.102 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-19T06:06:42.130 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-19T06:06:42.155 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-19T06:06:42.186 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-19T06:06:42.210 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-19T06:06:42.398 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-19T06:06:42.452 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-19T06:06:42.868 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-19T06:06:42.924 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 7/34 2024-09-19T06:06:43.000 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 7/34 2024-09-19T06:06:43.090 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 8/34 2024-09-19T06:06:43.116 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/34 2024-09-19T06:06:43.179 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/34 2024-09-19T06:06:43.217 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-19T06:06:43.261 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x 11/34 2024-09-19T06:06:43.332 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 12/34 2024-09-19T06:06:43.360 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 13/34 2024-09-19T06:06:43.425 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 13/34 2024-09-19T06:06:43.459 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-19T06:06:43.572 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-19T06:06:43.601 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-19T06:06:43.662 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-19T06:06:43.700 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-19T06:06:43.738 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-19T06:06:43.770 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-19T06:06:44.101 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-19T06:06:44.149 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 22/34 2024-09-19T06:06:44.241 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 22/34 2024-09-19T06:06:44.288 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 23/34 2024-09-19T06:06:44.332 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-19T06:06:44.851 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-19T06:06:44.877 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 26/34 2024-09-19T06:06:44.959 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 26/34 2024-09-19T06:06:45.037 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 27/34 2024-09-19T06:06:45.071 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86 28/34 2024-09-19T06:06:45.734 INFO:teuthology.orchestra.run.smithi155.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-19T06:06:47.513 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 29/34 2024-09-19T06:06:47.550 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 29/34 2024-09-19T06:06:47.977 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-09-19T06:06:47.996 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:06:47.996 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-09-19T06:06:47.996 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:06:47.996 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-09-19T06:06:47.996 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T06:06:47.996 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrading: 2024-09-19T06:06:47.997 INFO:teuthology.orchestra.run.smithi155.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T06:06:47.997 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T06:06:47.997 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-09-19T06:06:47.997 INFO:teuthology.orchestra.run.smithi155.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T06:06:47.997 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T06:06:47.997 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T06:06:47.997 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T06:06:47.997 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T06:06:47.997 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T06:06:47.997 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T06:06:47.997 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T06:06:47.998 INFO:teuthology.orchestra.run.smithi155.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T06:06:47.998 INFO:teuthology.orchestra.run.smithi155.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T06:06:47.998 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T06:06:47.998 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T06:06:47.998 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T06:06:47.998 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T06:06:47.998 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T06:06:47.998 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T06:06:47.998 INFO:teuthology.orchestra.run.smithi155.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T06:06:47.998 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T06:06:47.998 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T06:06:47.998 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T06:06:47.999 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T06:06:47.999 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T06:06:47.999 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T06:06:47.999 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T06:06:47.999 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T06:06:47.999 INFO:teuthology.orchestra.run.smithi155.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T06:06:47.999 INFO:teuthology.orchestra.run.smithi155.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T06:06:47.999 INFO:teuthology.orchestra.run.smithi155.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T06:06:47.999 INFO:teuthology.orchestra.run.smithi155.stdout:Installing weak dependencies: 2024-09-19T06:06:47.999 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T06:06:47.999 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:06:47.999 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-09-19T06:06:47.999 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:06:47.999 INFO:teuthology.orchestra.run.smithi155.stdout:Install 30 Packages 2024-09-19T06:06:48.000 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrade 2 Packages 2024-09-19T06:06:48.000 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:06:48.000 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 60 M 2024-09-19T06:06:48.000 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-09-19T06:06:48.138 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 29/34 2024-09-19T06:06:48.171 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 30/34 2024-09-19T06:06:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 30/34 2024-09-19T06:06:48.271 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T06:06:48.272 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T06:06:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:06:48.313 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 31/34 2024-09-19T06:06:48.777 INFO:teuthology.orchestra.run.smithi155.stdout:(1/32): ceph-base-19.1.1-361.g2f599a46.el9.x86_ 11 MB/s | 5.4 MB 00:00 2024-09-19T06:06:48.844 INFO:teuthology.orchestra.run.smithi155.stdout:(2/32): ceph-selinux-19.1.1-361.g2f599a46.el9.x 378 kB/s | 25 kB 00:00 2024-09-19T06:06:48.969 INFO:teuthology.orchestra.run.smithi155.stdout:(3/32): ceph-radosgw-19.1.1-361.g2f599a46.el9.x 16 MB/s | 11 MB 00:00 2024-09-19T06:06:49.003 INFO:teuthology.orchestra.run.smithi155.stdout:(4/32): libcephfs2-19.1.1-361.g2f599a46.el9.x86 4.6 MB/s | 751 kB 00:00 2024-09-19T06:06:49.028 INFO:teuthology.orchestra.run.smithi155.stdout:(5/32): libradosstriper1-19.1.1-361.g2f599a46.e 8.5 MB/s | 506 kB 00:00 2024-09-19T06:06:49.095 INFO:teuthology.orchestra.run.smithi155.stdout:(6/32): python3-ceph-argparse-19.1.1-361.g2f599 675 kB/s | 45 kB 00:00 2024-09-19T06:06:49.195 INFO:teuthology.orchestra.run.smithi155.stdout:(7/32): python3-ceph-common-19.1.1-361.g2f599a4 1.3 MB/s | 136 kB 00:00 2024-09-19T06:06:49.279 INFO:teuthology.orchestra.run.smithi155.stdout:(8/32): librgw2-19.1.1-361.g2f599a46.el9.x86_64 19 MB/s | 5.3 MB 00:00 2024-09-19T06:06:49.304 INFO:teuthology.orchestra.run.smithi155.stdout:(9/32): python3-cephfs-19.1.1-361.g2f599a46.el9 1.5 MB/s | 161 kB 00:00 2024-09-19T06:06:49.329 INFO:teuthology.orchestra.run.smithi155.stdout:(10/32): python3-rados-19.1.1-361.g2f599a46.el9 6.2 MB/s | 320 kB 00:00 2024-09-19T06:06:49.363 INFO:teuthology.orchestra.run.smithi155.stdout:(11/32): python3-rbd-19.1.1-361.g2f599a46.el9.x 5.0 MB/s | 300 kB 00:00 2024-09-19T06:06:49.388 INFO:teuthology.orchestra.run.smithi155.stdout:(12/32): python3-rgw-19.1.1-361.g2f599a46.el9.x 1.7 MB/s | 99 kB 00:00 2024-09-19T06:06:49.630 INFO:teuthology.orchestra.run.smithi155.stdout:(13/32): ceph-common-19.1.1-361.g2f599a46.el9.x 16 MB/s | 21 MB 00:01 2024-09-19T06:06:49.672 INFO:teuthology.orchestra.run.smithi155.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 346 kB/s | 107 kB 00:00 2024-09-19T06:06:49.698 INFO:teuthology.orchestra.run.smithi155.stdout:(15/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 519 kB/s | 160 kB 00:00 2024-09-19T06:06:49.748 INFO:teuthology.orchestra.run.smithi155.stdout:(16/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 387 kB/s | 45 kB 00:00 2024-09-19T06:06:49.774 INFO:teuthology.orchestra.run.smithi155.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-19T06:06:49.807 INFO:teuthology.orchestra.run.smithi155.stdout:(18/32): librdkafka-1.6.1-102.el9.x86_64.rpm 4.8 MB/s | 662 kB 00:00 2024-09-19T06:06:49.833 INFO:teuthology.orchestra.run.smithi155.stdout:(19/32): python3-prettytable-0.7.2-27.el9.noarc 501 kB/s | 42 kB 00:00 2024-09-19T06:06:49.858 INFO:teuthology.orchestra.run.smithi155.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 715 kB/s | 36 kB 00:00 2024-09-19T06:06:49.883 INFO:teuthology.orchestra.run.smithi155.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-09-19T06:06:49.908 INFO:teuthology.orchestra.run.smithi155.stdout:(22/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.6 MB/s | 221 kB 00:00 2024-09-19T06:06:49.975 INFO:teuthology.orchestra.run.smithi155.stdout:(23/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.6 MB/s | 308 kB 00:00 2024-09-19T06:06:50.000 INFO:teuthology.orchestra.run.smithi155.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 2.0 MB/s | 49 kB 00:00 2024-09-19T06:06:50.026 INFO:teuthology.orchestra.run.smithi155.stdout:(25/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 254 kB/s | 26 kB 00:00 2024-09-19T06:06:50.051 INFO:teuthology.orchestra.run.smithi155.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-19T06:06:50.093 INFO:teuthology.orchestra.run.smithi155.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-19T06:06:50.143 INFO:teuthology.orchestra.run.smithi155.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-09-19T06:06:50.185 INFO:teuthology.orchestra.run.smithi155.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-19T06:06:50.261 INFO:teuthology.orchestra.run.smithi155.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-19T06:06:50.394 INFO:teuthology.orchestra.run.smithi155.stdout:(31/32): librados2-19.1.1-361.g2f599a46.el9.x86 14 MB/s | 3.4 MB 00:00 2024-09-19T06:06:50.452 INFO:teuthology.orchestra.run.smithi155.stdout:(32/32): librbd1-19.1.1-361.g2f599a46.el9.x86_6 12 MB/s | 3.2 MB 00:00 2024-09-19T06:06:50.458 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:06:50.458 INFO:teuthology.orchestra.run.smithi155.stdout:Total 25 MB/s | 60 MB 00:02 2024-09-19T06:06:50.555 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-09-19T06:06:50.592 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-09-19T06:06:50.593 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-09-19T06:06:51.278 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-09-19T06:06:51.279 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-09-19T06:06:52.787 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-09-19T06:06:52.886 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-19T06:06:52.924 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-19T06:06:52.950 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-19T06:06:52.972 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-19T06:06:53.000 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-19T06:06:53.199 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-19T06:06:53.253 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-19T06:06:53.590 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-19T06:06:53.702 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 7/34 2024-09-19T06:06:53.794 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 7/34 2024-09-19T06:06:53.884 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 8/34 2024-09-19T06:06:53.928 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/34 2024-09-19T06:06:53.999 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/34 2024-09-19T06:06:54.037 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-19T06:06:54.081 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x 11/34 2024-09-19T06:06:54.151 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 12/34 2024-09-19T06:06:54.180 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 13/34 2024-09-19T06:06:54.245 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 13/34 2024-09-19T06:06:54.279 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-19T06:06:54.401 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-19T06:06:54.439 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-19T06:06:54.489 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-19T06:06:54.521 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-19T06:06:54.550 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-19T06:06:54.583 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-19T06:06:54.897 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-19T06:06:54.946 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 22/34 2024-09-19T06:06:55.036 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 22/34 2024-09-19T06:06:55.067 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 23/34 2024-09-19T06:06:55.112 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-19T06:06:55.602 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-19T06:06:55.632 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 26/34 2024-09-19T06:06:55.694 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 26/34 2024-09-19T06:06:55.772 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 27/34 2024-09-19T06:06:55.801 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86 28/34 2024-09-19T06:06:58.114 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 29/34 2024-09-19T06:06:58.148 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 29/34 2024-09-19T06:06:58.733 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 29/34 2024-09-19T06:06:58.777 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 30/34 2024-09-19T06:06:58.872 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 30/34 2024-09-19T06:06:58.872 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T06:06:58.872 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T06:06:58.872 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:06:58.903 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 31/34 2024-09-19T06:06:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 31/34 2024-09-19T06:06:59.017 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /sys 2024-09-19T06:06:59.017 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /proc 2024-09-19T06:06:59.017 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /mnt 2024-09-19T06:06:59.017 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /var/tmp 2024-09-19T06:06:59.017 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /home 2024-09-19T06:06:59.017 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /root 2024-09-19T06:06:59.018 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /tmp 2024-09-19T06:06:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:07:00.045 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 32/34 2024-09-19T06:07:00.073 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 32/34 2024-09-19T06:07:00.073 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:07:00.073 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T06:07:00.073 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T06:07:00.073 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T06:07:00.073 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:07:00.075 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-19T06:07:00.164 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-19T06:07:00.164 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-19T06:07:01.366 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-19T06:07:01.366 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 1/34 2024-09-19T06:07:01.366 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2/34 2024-09-19T06:07:01.366 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 3/34 2024-09-19T06:07:01.366 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 4/34 2024-09-19T06:07:01.366 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 5/34 2024-09-19T06:07:01.367 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 6/34 2024-09-19T06:07:01.367 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 7/34 2024-09-19T06:07:01.367 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x 8/34 2024-09-19T06:07:01.367 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86 9/34 2024-09-19T06:07:01.367 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 10/34 2024-09-19T06:07:01.367 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 11/34 2024-09-19T06:07:01.367 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 12/34 2024-09-19T06:07:01.367 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 13/34 2024-09-19T06:07:01.367 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-19T06:07:01.367 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-19T06:07:01.367 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-19T06:07:01.367 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-19T06:07:01.367 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-19T06:07:01.367 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-19T06:07:01.368 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-19T06:07:01.368 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-19T06:07:01.368 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-19T06:07:01.368 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-19T06:07:01.368 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-19T06:07:01.368 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-19T06:07:01.368 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-19T06:07:01.368 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-19T06:07:01.368 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-19T06:07:01.368 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-19T06:07:01.368 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-19T06:07:01.368 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 31/34 2024-09-19T06:07:01.368 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-19T06:07:01.369 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 33/34 2024-09-19T06:07:02.520 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-19T06:07:02.520 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:07:02.520 INFO:teuthology.orchestra.run.smithi053.stdout:Upgraded: 2024-09-19T06:07:02.521 INFO:teuthology.orchestra.run.smithi053.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:02.521 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:02.521 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-19T06:07:02.521 INFO:teuthology.orchestra.run.smithi053.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T06:07:02.521 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:02.521 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:02.521 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:02.521 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:02.521 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T06:07:02.521 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T06:07:02.521 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T06:07:02.521 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:02.521 INFO:teuthology.orchestra.run.smithi053.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T06:07:02.522 INFO:teuthology.orchestra.run.smithi053.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T06:07:02.522 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T06:07:02.522 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:02.522 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T06:07:02.522 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:02.522 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T06:07:02.522 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T06:07:02.522 INFO:teuthology.orchestra.run.smithi053.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T06:07:02.522 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:02.522 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:02.522 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:02.522 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T06:07:02.522 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:02.522 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:02.522 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:02.523 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T06:07:02.523 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T06:07:02.523 INFO:teuthology.orchestra.run.smithi053.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T06:07:02.523 INFO:teuthology.orchestra.run.smithi053.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T06:07:02.523 INFO:teuthology.orchestra.run.smithi053.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T06:07:02.523 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:07:02.523 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-19T06:07:02.896 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-test 2024-09-19T06:07:03.491 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:00:29 ago on Thu 19 Sep 2024 06:06:34 AM UTC. 2024-09-19T06:07:03.639 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-19T06:07:03.640 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:07:03.640 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-19T06:07:03.640 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:07:03.640 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-19T06:07:03.640 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T06:07:03.640 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-19T06:07:03.640 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T06:07:03.641 INFO:teuthology.orchestra.run.smithi053.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T06:07:03.641 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T06:07:03.641 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:07:03.641 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-19T06:07:03.641 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:07:03.641 INFO:teuthology.orchestra.run.smithi053.stdout:Install 4 Packages 2024-09-19T06:07:03.641 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:07:03.642 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 49 M 2024-09-19T06:07:03.642 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 209 M 2024-09-19T06:07:03.642 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-19T06:07:04.178 INFO:teuthology.orchestra.run.smithi053.stdout:(1/4): libcephsqlite-19.1.1-361.g2f599a46.el9.x 638 kB/s | 163 kB 00:00 2024-09-19T06:07:04.553 INFO:teuthology.orchestra.run.smithi053.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 170 kB/s | 64 kB 00:00 2024-09-19T06:07:04.587 INFO:teuthology.orchestra.run.smithi053.stdout:(3/4): socat-1.7.4.1-6.el9.x86_64.rpm 457 kB/s | 304 kB 00:00 2024-09-19T06:07:06.729 INFO:teuthology.orchestra.run.smithi053.stdout:(4/4): ceph-test-19.1.1-361.g2f599a46.el9.x86_6 17 MB/s | 49 MB 00:02 2024-09-19T06:07:06.731 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:07:06.731 INFO:teuthology.orchestra.run.smithi053.stdout:Total 16 MB/s | 49 MB 00:03 2024-09-19T06:07:06.759 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-19T06:07:06.782 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-19T06:07:06.782 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-19T06:07:07.185 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-19T06:07:07.185 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-19T06:07:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-19T06:07:07.822 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-19T06:07:07.862 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-19T06:07:07.889 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 3/4 2024-09-19T06:07:08.976 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 31/34 2024-09-19T06:07:08.977 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /sys 2024-09-19T06:07:08.977 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /proc 2024-09-19T06:07:08.977 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /mnt 2024-09-19T06:07:08.977 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /var/tmp 2024-09-19T06:07:08.977 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /home 2024-09-19T06:07:08.977 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /root 2024-09-19T06:07:08.977 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /tmp 2024-09-19T06:07:08.977 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:07:09.959 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 32/34 2024-09-19T06:07:09.984 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 32/34 2024-09-19T06:07:09.984 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:07:09.984 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T06:07:09.984 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T06:07:09.985 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T06:07:09.985 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:07:09.986 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-19T06:07:10.070 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-19T06:07:10.070 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-19T06:07:11.262 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-19T06:07:11.262 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 1/34 2024-09-19T06:07:11.262 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2/34 2024-09-19T06:07:11.262 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 3/34 2024-09-19T06:07:11.262 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 4/34 2024-09-19T06:07:11.262 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 5/34 2024-09-19T06:07:11.263 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 6/34 2024-09-19T06:07:11.263 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 7/34 2024-09-19T06:07:11.263 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x 8/34 2024-09-19T06:07:11.263 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86 9/34 2024-09-19T06:07:11.263 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 10/34 2024-09-19T06:07:11.263 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 11/34 2024-09-19T06:07:11.263 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 12/34 2024-09-19T06:07:11.263 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 13/34 2024-09-19T06:07:11.263 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-19T06:07:11.263 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-19T06:07:11.263 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-19T06:07:11.263 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-19T06:07:11.263 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-19T06:07:11.263 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-19T06:07:11.263 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-19T06:07:11.264 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-19T06:07:11.264 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-19T06:07:11.264 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-19T06:07:11.264 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-19T06:07:11.264 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-19T06:07:11.264 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-19T06:07:11.264 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-19T06:07:11.264 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-19T06:07:11.264 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-19T06:07:11.264 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-19T06:07:11.264 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 31/34 2024-09-19T06:07:11.264 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-19T06:07:11.264 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 33/34 2024-09-19T06:07:12.422 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-19T06:07:12.423 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:07:12.423 INFO:teuthology.orchestra.run.smithi155.stdout:Upgraded: 2024-09-19T06:07:12.423 INFO:teuthology.orchestra.run.smithi155.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:12.423 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:12.423 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-09-19T06:07:12.423 INFO:teuthology.orchestra.run.smithi155.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T06:07:12.423 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:12.423 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:12.423 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:12.423 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:12.423 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T06:07:12.423 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T06:07:12.423 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T06:07:12.423 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:12.424 INFO:teuthology.orchestra.run.smithi155.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T06:07:12.424 INFO:teuthology.orchestra.run.smithi155.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T06:07:12.424 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T06:07:12.424 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:12.424 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T06:07:12.424 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:12.424 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T06:07:12.424 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T06:07:12.424 INFO:teuthology.orchestra.run.smithi155.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T06:07:12.424 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:12.424 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:12.424 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:12.424 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T06:07:12.425 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:12.425 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:12.425 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:12.425 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T06:07:12.425 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T06:07:12.425 INFO:teuthology.orchestra.run.smithi155.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T06:07:12.425 INFO:teuthology.orchestra.run.smithi155.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T06:07:12.425 INFO:teuthology.orchestra.run.smithi155.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T06:07:12.425 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:07:12.425 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-09-19T06:07:12.760 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 3/4 2024-09-19T06:07:12.828 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-test 2024-09-19T06:07:12.839 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 4/4 2024-09-19T06:07:13.405 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:00:28 ago on Thu 19 Sep 2024 06:06:45 AM UTC. 2024-09-19T06:07:13.542 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-09-19T06:07:13.542 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:07:13.542 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-09-19T06:07:13.542 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:07:13.542 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-09-19T06:07:13.543 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T06:07:13.543 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-09-19T06:07:13.543 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T06:07:13.543 INFO:teuthology.orchestra.run.smithi155.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T06:07:13.543 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T06:07:13.543 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:07:13.543 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-09-19T06:07:13.543 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:07:13.543 INFO:teuthology.orchestra.run.smithi155.stdout:Install 4 Packages 2024-09-19T06:07:13.543 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:07:13.544 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 49 M 2024-09-19T06:07:13.544 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 209 M 2024-09-19T06:07:13.544 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-09-19T06:07:13.885 INFO:teuthology.orchestra.run.smithi155.stdout:(1/4): libcephsqlite-19.1.1-361.g2f599a46.el9.x 778 kB/s | 163 kB 00:00 2024-09-19T06:07:13.952 INFO:teuthology.orchestra.run.smithi155.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.1 MB/s | 304 kB 00:00 2024-09-19T06:07:14.069 INFO:teuthology.orchestra.run.smithi155.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 347 kB/s | 64 kB 00:00 2024-09-19T06:07:15.158 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 4/4 2024-09-19T06:07:15.158 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 1/4 2024-09-19T06:07:15.158 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2/4 2024-09-19T06:07:15.158 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-19T06:07:15.228 INFO:teuthology.orchestra.run.smithi155.stdout:(4/4): ceph-test-19.1.1-361.g2f599a46.el9.x86_6 31 MB/s | 49 MB 00:01 2024-09-19T06:07:15.230 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:07:15.230 INFO:teuthology.orchestra.run.smithi155.stdout:Total 29 MB/s | 49 MB 00:01 2024-09-19T06:07:15.258 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-09-19T06:07:15.278 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-09-19T06:07:15.278 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-09-19T06:07:15.505 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-19T06:07:15.505 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:07:15.505 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-19T06:07:15.506 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:15.506 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:15.506 INFO:teuthology.orchestra.run.smithi053.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T06:07:15.506 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T06:07:15.506 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:07:15.506 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-19T06:07:15.667 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-09-19T06:07:15.667 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-09-19T06:07:15.784 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph 2024-09-19T06:07:16.197 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-09-19T06:07:16.287 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-19T06:07:16.335 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-19T06:07:16.363 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 3/4 2024-09-19T06:07:16.392 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:00:42 ago on Thu 19 Sep 2024 06:06:34 AM UTC. 2024-09-19T06:07:16.533 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-19T06:07:16.535 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:07:16.536 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-19T06:07:16.536 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:07:16.536 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-19T06:07:16.536 INFO:teuthology.orchestra.run.smithi053.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T06:07:16.537 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-19T06:07:16.537 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T06:07:16.537 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T06:07:16.537 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T06:07:16.537 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T06:07:16.537 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T06:07:16.537 INFO:teuthology.orchestra.run.smithi053.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T06:07:16.538 INFO:teuthology.orchestra.run.smithi053.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T06:07:16.538 INFO:teuthology.orchestra.run.smithi053.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T06:07:16.538 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T06:07:16.538 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T06:07:16.538 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T06:07:16.538 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T06:07:16.538 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T06:07:16.538 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T06:07:16.538 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T06:07:16.538 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T06:07:16.538 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T06:07:16.538 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T06:07:16.539 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T06:07:16.539 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T06:07:16.539 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T06:07:16.539 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T06:07:16.539 INFO:teuthology.orchestra.run.smithi053.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T06:07:16.539 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T06:07:16.539 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T06:07:16.539 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T06:07:16.539 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T06:07:16.539 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T06:07:16.539 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T06:07:16.539 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T06:07:16.540 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T06:07:16.540 INFO:teuthology.orchestra.run.smithi053.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T06:07:16.540 INFO:teuthology.orchestra.run.smithi053.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T06:07:16.540 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T06:07:16.540 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T06:07:16.540 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T06:07:16.540 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:07:16.540 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-19T06:07:16.540 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:07:16.540 INFO:teuthology.orchestra.run.smithi053.stdout:Install 37 Packages 2024-09-19T06:07:16.540 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:07:16.541 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 30 M 2024-09-19T06:07:16.541 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 108 M 2024-09-19T06:07:16.541 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-19T06:07:17.422 INFO:teuthology.orchestra.run.smithi053.stdout:(1/37): ceph-19.1.1-361.g2f599a46.el9.x86_64.rp 47 kB/s | 6.6 kB 00:00 2024-09-19T06:07:17.638 INFO:teuthology.orchestra.run.smithi053.stdout:(2/37): ceph-mds-19.1.1-361.g2f599a46.el9.x86_6 6.7 MB/s | 2.4 MB 00:00 2024-09-19T06:07:17.680 INFO:teuthology.orchestra.run.smithi053.stdout:(3/37): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_6 3.8 MB/s | 1.5 MB 00:00 2024-09-19T06:07:17.780 INFO:teuthology.orchestra.run.smithi053.stdout:(4/37): ceph-mgr-modules-core-19.1.1-361.g2f599 2.4 MB/s | 249 kB 00:00 2024-09-19T06:07:17.856 INFO:teuthology.orchestra.run.smithi053.stdout:(5/37): ceph-mon-19.1.1-361.g2f599a46.el9.x86_6 11 MB/s | 4.7 MB 00:00 2024-09-19T06:07:17.989 INFO:teuthology.orchestra.run.smithi053.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-09-19T06:07:18.034 INFO:teuthology.orchestra.run.smithi053.stdout:(7/37): python3-cryptography-36.0.1-4.el9.x86_6 6.9 MB/s | 1.2 MB 00:00 2024-09-19T06:07:18.193 INFO:teuthology.orchestra.run.smithi053.stdout:(8/37): ceph-osd-19.1.1-361.g2f599a46.el9.x86_6 30 MB/s | 17 MB 00:00 2024-09-19T06:07:18.218 INFO:teuthology.orchestra.run.smithi053.stdout:(9/37): python3-ply-3.11-14.el9.noarch.rpm 466 kB/s | 106 kB 00:00 2024-09-19T06:07:18.244 INFO:teuthology.orchestra.run.smithi053.stdout:(10/37): python3-pycparser-2.20-6.el9.noarch.rp 647 kB/s | 135 kB 00:00 2024-09-19T06:07:18.310 INFO:teuthology.orchestra.run.smithi053.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 2.3 MB/s | 215 kB 00:00 2024-09-19T06:07:18.336 INFO:teuthology.orchestra.run.smithi053.stdout:(12/37): python3-requests-2.25.1-8.el9.noarch.r 880 kB/s | 125 kB 00:00 2024-09-19T06:07:18.369 INFO:teuthology.orchestra.run.smithi053.stdout:(13/37): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-09-19T06:07:18.411 INFO:teuthology.orchestra.run.smithi053.stdout:(14/37): python3-packaging-20.9-5.el9.noarch.rp 1.8 MB/s | 77 kB 00:00 2024-09-19T06:07:18.436 INFO:teuthology.orchestra.run.smithi053.stdout:(15/37): python3-markupsafe-1.1.1-12.el9.x86_64 346 kB/s | 35 kB 00:00 2024-09-19T06:07:18.462 INFO:teuthology.orchestra.run.smithi053.stdout:(16/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-19T06:07:18.487 INFO:teuthology.orchestra.run.smithi053.stdout:(17/37): python3-mako-1.1.4-6.el9.noarch.rpm 978 kB/s | 172 kB 00:00 2024-09-19T06:07:18.512 INFO:teuthology.orchestra.run.smithi053.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 553 kB/s | 42 kB 00:00 2024-09-19T06:07:18.537 INFO:teuthology.orchestra.run.smithi053.stdout:(19/37): lua-devel-5.4.4-4.el9.x86_64.rpm 296 kB/s | 22 kB 00:00 2024-09-19T06:07:18.587 INFO:teuthology.orchestra.run.smithi053.stdout:(20/37): luarocks-3.9.2-5.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-09-19T06:07:18.613 INFO:teuthology.orchestra.run.smithi053.stdout:(21/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 580 kB/s | 43 kB 00:00 2024-09-19T06:07:18.638 INFO:teuthology.orchestra.run.smithi053.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 430 kB/s | 11 kB 00:00 2024-09-19T06:07:18.663 INFO:teuthology.orchestra.run.smithi053.stdout:(23/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-19T06:07:18.688 INFO:teuthology.orchestra.run.smithi053.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 353 kB/s | 18 kB 00:00 2024-09-19T06:07:18.714 INFO:teuthology.orchestra.run.smithi053.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 461 kB/s | 23 kB 00:00 2024-09-19T06:07:18.739 INFO:teuthology.orchestra.run.smithi053.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-19T06:07:18.764 INFO:teuthology.orchestra.run.smithi053.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-09-19T06:07:18.789 INFO:teuthology.orchestra.run.smithi053.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 927 kB/s | 46 kB 00:00 2024-09-19T06:07:18.815 INFO:teuthology.orchestra.run.smithi053.stdout:(29/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.6 MB/s | 358 kB 00:00 2024-09-19T06:07:18.840 INFO:teuthology.orchestra.run.smithi053.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-19T06:07:18.865 INFO:teuthology.orchestra.run.smithi053.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 329 kB/s | 16 kB 00:00 2024-09-19T06:07:18.882 INFO:teuthology.orchestra.run.smithi053.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 2.1 MB/s | 36 kB 00:00 2024-09-19T06:07:18.908 INFO:teuthology.orchestra.run.smithi053.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.3 MB/s | 90 kB 00:00 2024-09-19T06:07:18.933 INFO:teuthology.orchestra.run.smithi053.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-09-19T06:07:18.958 INFO:teuthology.orchestra.run.smithi053.stdout:(35/37): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-19T06:07:18.992 INFO:teuthology.orchestra.run.smithi053.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 341 kB/s | 20 kB 00:00 2024-09-19T06:07:19.017 INFO:teuthology.orchestra.run.smithi053.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-19T06:07:19.022 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:07:19.022 INFO:teuthology.orchestra.run.smithi053.stdout:Total 12 MB/s | 30 MB 00:02 2024-09-19T06:07:19.183 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-19T06:07:19.228 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-19T06:07:19.229 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-19T06:07:19.725 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-19T06:07:19.725 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-19T06:07:20.993 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 3/4 2024-09-19T06:07:21.077 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 4/4 2024-09-19T06:07:21.123 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-19T06:07:21.187 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-19T06:07:21.220 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-19T06:07:21.247 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-19T06:07:21.324 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-19T06:07:21.372 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-19T06:07:21.405 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-19T06:07:21.501 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-19T06:07:21.578 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-19T06:07:21.624 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-19T06:07:21.659 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-19T06:07:21.688 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-19T06:07:21.714 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-19T06:07:21.748 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-19T06:07:21.789 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-19T06:07:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-19T06:07:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-19T06:07:21.923 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-19T06:07:21.968 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-19T06:07:22.048 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-19T06:07:22.201 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-19T06:07:22.291 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-19T06:07:22.358 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-19T06:07:22.395 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-19T06:07:22.454 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-19T06:07:22.517 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-19T06:07:22.716 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-19T06:07:22.766 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-19T06:07:22.848 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-19T06:07:22.948 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-19T06:07:23.012 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-19T06:07:23.137 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-19T06:07:23.320 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.n 32/37 2024-09-19T06:07:23.349 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 33/37 2024-09-19T06:07:23.377 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 33/37 2024-09-19T06:07:23.377 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:07:23.377 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T06:07:23.377 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T06:07:23.377 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T06:07:23.377 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:07:23.413 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 4/4 2024-09-19T06:07:23.414 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 1/4 2024-09-19T06:07:23.414 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2/4 2024-09-19T06:07:23.414 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-19T06:07:23.739 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-19T06:07:23.740 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:07:23.740 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-09-19T06:07:23.740 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:23.740 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:23.740 INFO:teuthology.orchestra.run.smithi155.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T06:07:23.740 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T06:07:23.740 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:07:23.740 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-09-19T06:07:24.039 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph 2024-09-19T06:07:24.627 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:00:39 ago on Thu 19 Sep 2024 06:06:45 AM UTC. 2024-09-19T06:07:24.769 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-09-19T06:07:24.772 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:07:24.772 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-09-19T06:07:24.772 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:07:24.772 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-09-19T06:07:24.772 INFO:teuthology.orchestra.run.smithi155.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T06:07:24.772 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-09-19T06:07:24.773 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T06:07:24.773 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T06:07:24.773 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T06:07:24.773 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T06:07:24.773 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T06:07:24.773 INFO:teuthology.orchestra.run.smithi155.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T06:07:24.773 INFO:teuthology.orchestra.run.smithi155.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T06:07:24.773 INFO:teuthology.orchestra.run.smithi155.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T06:07:24.773 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T06:07:24.773 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T06:07:24.773 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T06:07:24.773 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T06:07:24.773 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T06:07:24.774 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T06:07:24.774 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T06:07:24.774 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T06:07:24.774 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T06:07:24.774 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T06:07:24.774 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T06:07:24.774 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T06:07:24.774 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T06:07:24.774 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T06:07:24.774 INFO:teuthology.orchestra.run.smithi155.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T06:07:24.774 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T06:07:24.774 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T06:07:24.774 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T06:07:24.774 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T06:07:24.775 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T06:07:24.775 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T06:07:24.775 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T06:07:24.775 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T06:07:24.775 INFO:teuthology.orchestra.run.smithi155.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T06:07:24.775 INFO:teuthology.orchestra.run.smithi155.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T06:07:24.775 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T06:07:24.775 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T06:07:24.775 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T06:07:24.775 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:07:24.775 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-09-19T06:07:24.775 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:07:24.776 INFO:teuthology.orchestra.run.smithi155.stdout:Install 37 Packages 2024-09-19T06:07:24.776 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:07:24.776 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 30 M 2024-09-19T06:07:24.776 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 108 M 2024-09-19T06:07:24.776 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-09-19T06:07:24.916 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 34/37 2024-09-19T06:07:24.945 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 34/37 2024-09-19T06:07:24.945 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:07:24.945 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T06:07:24.945 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T06:07:24.945 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T06:07:24.945 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:07:25.367 INFO:teuthology.orchestra.run.smithi155.stdout:(1/37): ceph-19.1.1-361.g2f599a46.el9.x86_64.rp 58 kB/s | 6.6 kB 00:00 2024-09-19T06:07:25.409 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 35/37 2024-09-19T06:07:25.434 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 35/37 2024-09-19T06:07:25.434 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:07:25.434 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T06:07:25.435 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T06:07:25.435 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T06:07:25.435 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:07:25.542 INFO:teuthology.orchestra.run.smithi155.stdout:(2/37): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_6 5.3 MB/s | 1.5 MB 00:00 2024-09-19T06:07:25.593 INFO:teuthology.orchestra.run.smithi155.stdout:(3/37): ceph-mds-19.1.1-361.g2f599a46.el9.x86_6 7.0 MB/s | 2.4 MB 00:00 2024-09-19T06:07:25.668 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 36/37 2024-09-19T06:07:25.693 INFO:teuthology.orchestra.run.smithi155.stdout:(4/37): ceph-mgr-modules-core-19.1.1-361.g2f599 2.4 MB/s | 249 kB 00:00 2024-09-19T06:07:25.693 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 36/37 2024-09-19T06:07:25.693 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:07:25.693 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T06:07:25.694 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T06:07:25.694 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T06:07:25.694 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:07:25.759 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 37/37 2024-09-19T06:07:25.777 INFO:teuthology.orchestra.run.smithi155.stdout:(5/37): ceph-mon-19.1.1-361.g2f599a46.el9.x86_6 12 MB/s | 4.7 MB 00:00 2024-09-19T06:07:25.919 INFO:teuthology.orchestra.run.smithi155.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-09-19T06:07:26.111 INFO:teuthology.orchestra.run.smithi155.stdout:(7/37): ceph-osd-19.1.1-361.g2f599a46.el9.x86_6 30 MB/s | 17 MB 00:00 2024-09-19T06:07:26.136 INFO:teuthology.orchestra.run.smithi155.stdout:(8/37): python3-ply-3.11-14.el9.noarch.rpm 491 kB/s | 106 kB 00:00 2024-09-19T06:07:26.186 INFO:teuthology.orchestra.run.smithi155.stdout:(9/37): python3-requests-2.25.1-8.el9.noarch.rp 2.4 MB/s | 125 kB 00:00 2024-09-19T06:07:26.229 INFO:teuthology.orchestra.run.smithi155.stdout:(10/37): python3-cryptography-36.0.1-4.el9.x86_ 2.7 MB/s | 1.2 MB 00:00 2024-09-19T06:07:26.253 INFO:teuthology.orchestra.run.smithi155.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 3.1 MB/s | 215 kB 00:00 2024-09-19T06:07:26.304 INFO:teuthology.orchestra.run.smithi155.stdout:(12/37): python3-pycparser-2.20-6.el9.noarch.rp 702 kB/s | 135 kB 00:00 2024-09-19T06:07:26.362 INFO:teuthology.orchestra.run.smithi155.stdout:(13/37): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-09-19T06:07:26.388 INFO:teuthology.orchestra.run.smithi155.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 416 kB/s | 35 kB 00:00 2024-09-19T06:07:26.413 INFO:teuthology.orchestra.run.smithi155.stdout:(15/37): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-19T06:07:26.430 INFO:teuthology.orchestra.run.smithi155.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 1.1 MB/s | 77 kB 00:00 2024-09-19T06:07:26.455 INFO:teuthology.orchestra.run.smithi155.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 759 kB/s | 51 kB 00:00 2024-09-19T06:07:26.481 INFO:teuthology.orchestra.run.smithi155.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 620 kB/s | 42 kB 00:00 2024-09-19T06:07:26.506 INFO:teuthology.orchestra.run.smithi155.stdout:(19/37): lua-devel-5.4.4-4.el9.x86_64.rpm 296 kB/s | 22 kB 00:00 2024-09-19T06:07:26.564 INFO:teuthology.orchestra.run.smithi155.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 746 kB/s | 43 kB 00:00 2024-09-19T06:07:26.581 INFO:teuthology.orchestra.run.smithi155.stdout:(21/37): luarocks-3.9.2-5.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-09-19T06:07:26.607 INFO:teuthology.orchestra.run.smithi155.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 431 kB/s | 11 kB 00:00 2024-09-19T06:07:26.632 INFO:teuthology.orchestra.run.smithi155.stdout:(23/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-19T06:07:26.657 INFO:teuthology.orchestra.run.smithi155.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 352 kB/s | 18 kB 00:00 2024-09-19T06:07:26.682 INFO:teuthology.orchestra.run.smithi155.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 461 kB/s | 23 kB 00:00 2024-09-19T06:07:26.708 INFO:teuthology.orchestra.run.smithi155.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-19T06:07:26.733 INFO:teuthology.orchestra.run.smithi155.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 391 kB/s | 20 kB 00:00 2024-09-19T06:07:26.758 INFO:teuthology.orchestra.run.smithi155.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 920 kB/s | 46 kB 00:00 2024-09-19T06:07:26.784 INFO:teuthology.orchestra.run.smithi155.stdout:(29/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.6 MB/s | 358 kB 00:00 2024-09-19T06:07:26.809 INFO:teuthology.orchestra.run.smithi155.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-19T06:07:26.834 INFO:teuthology.orchestra.run.smithi155.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 664 kB/s | 16 kB 00:00 2024-09-19T06:07:26.851 INFO:teuthology.orchestra.run.smithi155.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 2.1 MB/s | 90 kB 00:00 2024-09-19T06:07:26.876 INFO:teuthology.orchestra.run.smithi155.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 858 kB/s | 36 kB 00:00 2024-09-19T06:07:26.902 INFO:teuthology.orchestra.run.smithi155.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-09-19T06:07:26.927 INFO:teuthology.orchestra.run.smithi155.stdout:(35/37): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-19T06:07:26.952 INFO:teuthology.orchestra.run.smithi155.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 400 kB/s | 20 kB 00:00 2024-09-19T06:07:26.977 INFO:teuthology.orchestra.run.smithi155.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-09-19T06:07:26.982 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:07:26.982 INFO:teuthology.orchestra.run.smithi155.stdout:Total 14 MB/s | 30 MB 00:02 2024-09-19T06:07:27.134 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-09-19T06:07:27.175 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-09-19T06:07:27.175 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-09-19T06:07:27.538 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 37/37 2024-09-19T06:07:27.539 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/37 2024-09-19T06:07:27.539 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2/37 2024-09-19T06:07:27.539 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 3/37 2024-09-19T06:07:27.539 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 4/37 2024-09-19T06:07:27.539 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 5/37 2024-09-19T06:07:27.539 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.n 6/37 2024-09-19T06:07:27.539 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-19T06:07:27.539 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-19T06:07:27.539 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-19T06:07:27.539 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-19T06:07:27.539 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-19T06:07:27.539 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-19T06:07:27.540 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-19T06:07:27.540 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-19T06:07:27.540 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-19T06:07:27.540 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-19T06:07:27.540 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-19T06:07:27.540 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-19T06:07:27.540 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-19T06:07:27.540 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-19T06:07:27.540 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-19T06:07:27.540 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-19T06:07:27.540 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-19T06:07:27.540 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-19T06:07:27.540 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-19T06:07:27.541 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-19T06:07:27.541 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-19T06:07:27.541 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-19T06:07:27.542 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-19T06:07:27.543 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-19T06:07:27.543 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-19T06:07:27.543 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-19T06:07:27.543 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-19T06:07:27.543 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-19T06:07:27.543 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-19T06:07:27.544 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-19T06:07:27.692 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-09-19T06:07:27.692 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-09-19T06:07:28.506 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-19T06:07:28.506 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:07:28.506 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-19T06:07:28.506 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:28.506 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:28.506 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:28.507 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:07:28.507 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:28.507 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:28.507 INFO:teuthology.orchestra.run.smithi053.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T06:07:28.507 INFO:teuthology.orchestra.run.smithi053.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T06:07:28.507 INFO:teuthology.orchestra.run.smithi053.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T06:07:28.507 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T06:07:28.507 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T06:07:28.507 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T06:07:28.507 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T06:07:28.507 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T06:07:28.507 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T06:07:28.507 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T06:07:28.508 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T06:07:28.508 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T06:07:28.508 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T06:07:28.508 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T06:07:28.508 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T06:07:28.508 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T06:07:28.508 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T06:07:28.508 INFO:teuthology.orchestra.run.smithi053.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T06:07:28.508 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T06:07:28.508 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T06:07:28.508 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T06:07:28.508 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T06:07:28.508 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T06:07:28.509 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T06:07:28.509 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T06:07:28.509 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T06:07:28.509 INFO:teuthology.orchestra.run.smithi053.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T06:07:28.509 INFO:teuthology.orchestra.run.smithi053.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T06:07:28.509 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T06:07:28.509 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T06:07:28.509 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T06:07:28.509 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:07:28.509 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-19T06:07:28.894 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-base 2024-09-19T06:07:29.124 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-09-19T06:07:29.189 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-19T06:07:29.230 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-19T06:07:29.266 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-19T06:07:29.339 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-19T06:07:29.407 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-19T06:07:29.448 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-19T06:07:29.500 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:00:55 ago on Thu 19 Sep 2024 06:06:34 AM UTC. 2024-09-19T06:07:29.543 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-19T06:07:29.593 INFO:teuthology.orchestra.run.smithi053.stdout:Package ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 is already installed. 2024-09-19T06:07:29.628 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-19T06:07:29.637 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-19T06:07:29.638 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-19T06:07:29.638 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-19T06:07:29.676 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-19T06:07:29.711 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install cephadm 2024-09-19T06:07:29.711 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-19T06:07:29.753 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-19T06:07:29.796 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-19T06:07:29.830 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-19T06:07:29.879 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-19T06:07:29.922 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-19T06:07:29.956 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-19T06:07:30.005 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-19T06:07:30.046 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-19T06:07:30.119 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-19T06:07:30.257 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-19T06:07:30.310 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:00:56 ago on Thu 19 Sep 2024 06:06:34 AM UTC. 2024-09-19T06:07:30.337 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-19T06:07:30.398 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-19T06:07:30.437 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-19T06:07:30.449 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-19T06:07:30.449 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:07:30.449 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-19T06:07:30.450 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:07:30.450 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-19T06:07:30.450 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T06:07:30.450 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:07:30.450 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-19T06:07:30.450 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:07:30.450 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-09-19T06:07:30.450 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:07:30.450 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 765 k 2024-09-19T06:07:30.450 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 771 k 2024-09-19T06:07:30.451 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-19T06:07:30.494 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-19T06:07:30.550 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-19T06:07:30.699 INFO:teuthology.orchestra.run.smithi053.stdout:cephadm-19.1.1-361.g2f599a46.el9.noarch.rpm 3.0 MB/s | 765 kB 00:00 2024-09-19T06:07:30.700 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:07:30.700 INFO:teuthology.orchestra.run.smithi053.stdout:Total 3.0 MB/s | 765 kB 00:00 2024-09-19T06:07:30.700 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-19T06:07:30.706 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-19T06:07:30.706 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-19T06:07:30.712 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-19T06:07:30.713 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-19T06:07:30.754 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-19T06:07:30.789 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-19T06:07:30.808 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-19T06:07:30.886 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-19T06:07:30.976 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-19T06:07:31.021 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-19T06:07:31.141 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-19T06:07:31.271 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 1/1 2024-09-19T06:07:31.305 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.n 32/37 2024-09-19T06:07:31.343 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 33/37 2024-09-19T06:07:31.348 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 1/1 2024-09-19T06:07:31.369 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 33/37 2024-09-19T06:07:31.369 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:07:31.369 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T06:07:31.369 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T06:07:31.369 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T06:07:31.369 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:07:31.693 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 1/1 2024-09-19T06:07:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 1/1 2024-09-19T06:07:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:07:31.941 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-19T06:07:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:07:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:07:31.941 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-19T06:07:32.146 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-immutable-object-cache 2024-09-19T06:07:32.749 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:00:58 ago on Thu 19 Sep 2024 06:06:34 AM UTC. 2024-09-19T06:07:32.810 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 34/37 2024-09-19T06:07:32.838 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 34/37 2024-09-19T06:07:32.838 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:07:32.838 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T06:07:32.838 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T06:07:32.838 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T06:07:32.838 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:07:32.886 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-19T06:07:32.887 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:07:32.887 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repo Size 2024-09-19T06:07:32.887 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:07:32.887 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-19T06:07:32.888 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T06:07:32.888 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:07:32.888 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-19T06:07:32.888 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:07:32.888 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-09-19T06:07:32.888 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:07:32.888 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 145 k 2024-09-19T06:07:32.888 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 435 k 2024-09-19T06:07:32.888 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-19T06:07:33.074 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-immutable-object-cache-19.1.1-361.g2f599a4 784 kB/s | 145 kB 00:00 2024-09-19T06:07:33.074 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:07:33.074 INFO:teuthology.orchestra.run.smithi053.stdout:Total 777 kB/s | 145 kB 00:00 2024-09-19T06:07:33.075 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-19T06:07:33.083 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-19T06:07:33.083 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-19T06:07:33.143 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-19T06:07:33.143 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-19T06:07:33.286 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 35/37 2024-09-19T06:07:33.304 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-19T06:07:33.310 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 35/37 2024-09-19T06:07:33.310 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:07:33.310 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T06:07:33.310 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T06:07:33.310 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T06:07:33.310 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:07:33.416 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.e 1/1 2024-09-19T06:07:33.443 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.e 1/1 2024-09-19T06:07:33.443 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:07:33.443 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T06:07:33.443 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:07:33.537 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 36/37 2024-09-19T06:07:33.561 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 36/37 2024-09-19T06:07:33.561 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:07:33.561 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T06:07:33.561 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T06:07:33.561 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T06:07:33.561 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:07:33.625 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 37/37 2024-09-19T06:07:34.337 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.e 1/1 2024-09-19T06:07:34.337 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:07:34.337 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-19T06:07:34.337 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:34.338 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:07:34.338 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-19T06:07:34.561 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-mgr 2024-09-19T06:07:35.170 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:01 ago on Thu 19 Sep 2024 06:06:34 AM UTC. 2024-09-19T06:07:35.264 INFO:teuthology.orchestra.run.smithi053.stdout:Package ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 is already installed. 2024-09-19T06:07:35.309 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-19T06:07:35.310 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-19T06:07:35.310 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-19T06:07:35.385 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-mgr-dashboard 2024-09-19T06:07:35.433 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 37/37 2024-09-19T06:07:35.433 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/37 2024-09-19T06:07:35.433 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2/37 2024-09-19T06:07:35.433 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 3/37 2024-09-19T06:07:35.434 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 4/37 2024-09-19T06:07:35.434 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 5/37 2024-09-19T06:07:35.434 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.n 6/37 2024-09-19T06:07:35.434 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-19T06:07:35.434 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-19T06:07:35.434 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-19T06:07:35.434 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-19T06:07:35.434 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-19T06:07:35.434 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-19T06:07:35.434 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-19T06:07:35.434 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-19T06:07:35.434 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-19T06:07:35.434 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-19T06:07:35.435 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-19T06:07:35.435 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-19T06:07:35.435 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-19T06:07:35.435 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-19T06:07:35.435 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-19T06:07:35.435 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-19T06:07:35.435 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-19T06:07:35.435 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-19T06:07:35.435 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-19T06:07:35.435 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-19T06:07:35.435 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-19T06:07:35.436 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-19T06:07:35.437 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-19T06:07:35.437 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-19T06:07:35.437 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-19T06:07:35.437 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-19T06:07:35.437 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-19T06:07:35.437 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-19T06:07:35.437 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-19T06:07:35.437 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-19T06:07:35.993 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:01 ago on Thu 19 Sep 2024 06:06:34 AM UTC. 2024-09-19T06:07:36.130 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-19T06:07:36.131 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:07:36.131 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-19T06:07:36.131 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:07:36.131 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-19T06:07:36.131 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T06:07:36.131 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-19T06:07:36.131 INFO:teuthology.orchestra.run.smithi053.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T06:07:36.131 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T06:07:36.131 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T06:07:36.132 INFO:teuthology.orchestra.run.smithi053.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T06:07:36.132 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T06:07:36.132 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T06:07:36.132 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T06:07:36.132 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T06:07:36.132 INFO:teuthology.orchestra.run.smithi053.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T06:07:36.132 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T06:07:36.132 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T06:07:36.132 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:07:36.132 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-19T06:07:36.132 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:07:36.132 INFO:teuthology.orchestra.run.smithi053.stdout:Install 12 Packages 2024-09-19T06:07:36.132 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:07:36.133 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 8.7 M 2024-09-19T06:07:36.133 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 105 M 2024-09-19T06:07:36.133 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-19T06:07:36.517 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-19T06:07:36.517 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:07:36.518 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-09-19T06:07:36.518 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:36.518 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:36.518 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:36.518 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:07:36.518 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:36.518 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:36.518 INFO:teuthology.orchestra.run.smithi155.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T06:07:36.518 INFO:teuthology.orchestra.run.smithi155.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T06:07:36.518 INFO:teuthology.orchestra.run.smithi155.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T06:07:36.518 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T06:07:36.519 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T06:07:36.519 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T06:07:36.519 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T06:07:36.519 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T06:07:36.519 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T06:07:36.519 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T06:07:36.519 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T06:07:36.519 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T06:07:36.519 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T06:07:36.520 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T06:07:36.520 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T06:07:36.520 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T06:07:36.520 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T06:07:36.520 INFO:teuthology.orchestra.run.smithi155.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T06:07:36.520 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T06:07:36.520 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T06:07:36.520 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T06:07:36.520 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T06:07:36.520 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T06:07:36.520 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T06:07:36.521 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T06:07:36.521 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T06:07:36.521 INFO:teuthology.orchestra.run.smithi155.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T06:07:36.521 INFO:teuthology.orchestra.run.smithi155.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T06:07:36.521 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T06:07:36.521 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T06:07:36.521 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T06:07:36.521 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:07:36.521 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-09-19T06:07:36.769 INFO:teuthology.orchestra.run.smithi053.stdout:(1/12): ceph-prometheus-alerts-19.1.1-361.g2f59 105 kB/s | 16 kB 00:00 2024-09-19T06:07:36.794 INFO:teuthology.orchestra.run.smithi053.stdout:(2/12): ceph-grafana-dashboards-19.1.1-361.g2f5 174 kB/s | 32 kB 00:00 2024-09-19T06:07:36.914 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-base 2024-09-19T06:07:36.978 INFO:teuthology.orchestra.run.smithi053.stdout:(3/12): ceph-mgr-dashboard-19.1.1-361.g2f599a46 9.7 MB/s | 3.6 MB 00:00 2024-09-19T06:07:37.502 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:00:52 ago on Thu 19 Sep 2024 06:06:45 AM UTC. 2024-09-19T06:07:37.592 INFO:teuthology.orchestra.run.smithi155.stdout:Package ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 is already installed. 2024-09-19T06:07:37.633 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-09-19T06:07:37.634 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-09-19T06:07:37.635 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-09-19T06:07:37.670 INFO:teuthology.orchestra.run.smithi053.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 308 kB/s | 269 kB 00:00 2024-09-19T06:07:37.700 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install cephadm 2024-09-19T06:07:37.720 INFO:teuthology.orchestra.run.smithi053.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 11 MB/s | 548 kB 00:00 2024-09-19T06:07:37.762 INFO:teuthology.orchestra.run.smithi053.stdout:(6/12): protobuf-3.14.0-13.el9.x86_64.rpm 1.0 MB/s | 1.0 MB 00:00 2024-09-19T06:07:37.787 INFO:teuthology.orchestra.run.smithi053.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 291 kB/s | 19 kB 00:00 2024-09-19T06:07:37.812 INFO:teuthology.orchestra.run.smithi053.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 8.4 MB/s | 144 kB 00:00 2024-09-19T06:07:37.838 INFO:teuthology.orchestra.run.smithi053.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.3 MB/s | 31 kB 00:00 2024-09-19T06:07:37.871 INFO:teuthology.orchestra.run.smithi053.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 5.6 MB/s | 188 kB 00:00 2024-09-19T06:07:37.930 INFO:teuthology.orchestra.run.smithi053.stdout:(11/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 12 MB/s | 2.0 MB 00:00 2024-09-19T06:07:37.997 INFO:teuthology.orchestra.run.smithi053.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 847 kB/s | 863 kB 00:01 2024-09-19T06:07:38.001 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:07:38.001 INFO:teuthology.orchestra.run.smithi053.stdout:Total 4.6 MB/s | 8.7 MB 00:01 2024-09-19T06:07:38.056 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-19T06:07:38.075 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-19T06:07:38.075 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-19T06:07:38.279 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:00:53 ago on Thu 19 Sep 2024 06:06:45 AM UTC. 2024-09-19T06:07:38.294 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-19T06:07:38.294 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-19T06:07:38.410 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-09-19T06:07:38.411 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:07:38.411 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-09-19T06:07:38.411 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:07:38.411 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-09-19T06:07:38.412 INFO:teuthology.orchestra.run.smithi155.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T06:07:38.412 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:07:38.412 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-09-19T06:07:38.412 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:07:38.412 INFO:teuthology.orchestra.run.smithi155.stdout:Install 1 Package 2024-09-19T06:07:38.412 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:07:38.412 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 765 k 2024-09-19T06:07:38.412 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 771 k 2024-09-19T06:07:38.413 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-09-19T06:07:38.666 INFO:teuthology.orchestra.run.smithi155.stdout:cephadm-19.1.1-361.g2f599a46.el9.noarch.rpm 3.0 MB/s | 765 kB 00:00 2024-09-19T06:07:38.666 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:07:38.666 INFO:teuthology.orchestra.run.smithi155.stdout:Total 2.9 MB/s | 765 kB 00:00 2024-09-19T06:07:38.666 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-09-19T06:07:38.672 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-09-19T06:07:38.672 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-09-19T06:07:38.678 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-09-19T06:07:38.678 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-09-19T06:07:38.747 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-09-19T06:07:38.832 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-19T06:07:38.898 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-19T06:07:38.941 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-19T06:07:38.988 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-19T06:07:39.026 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-19T06:07:39.136 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-19T06:07:39.265 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-19T06:07:39.269 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 1/1 2024-09-19T06:07:39.357 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-19T06:07:39.361 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 1/1 2024-09-19T06:07:39.406 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-19T06:07:39.434 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-19T06:07:39.457 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9. 10/12 2024-09-19T06:07:39.748 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 1/1 2024-09-19T06:07:39.987 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 1/1 2024-09-19T06:07:39.987 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:07:39.987 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-09-19T06:07:39.987 INFO:teuthology.orchestra.run.smithi155.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:07:39.987 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:07:39.987 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-09-19T06:07:40.239 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-immutable-object-cache 2024-09-19T06:07:40.566 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9 11/12 2024-09-19T06:07:40.608 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noar 12/12 2024-09-19T06:07:40.825 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:00:55 ago on Thu 19 Sep 2024 06:06:45 AM UTC. 2024-09-19T06:07:40.961 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-09-19T06:07:40.961 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:07:40.961 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repo Size 2024-09-19T06:07:40.961 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:07:40.962 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-09-19T06:07:40.962 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T06:07:40.962 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:07:40.962 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-09-19T06:07:40.962 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:07:40.962 INFO:teuthology.orchestra.run.smithi155.stdout:Install 1 Package 2024-09-19T06:07:40.962 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:07:40.962 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 145 k 2024-09-19T06:07:40.962 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 435 k 2024-09-19T06:07:40.962 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-09-19T06:07:41.172 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-immutable-object-cache-19.1.1-361.g2f599a4 690 kB/s | 145 kB 00:00 2024-09-19T06:07:41.173 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:07:41.173 INFO:teuthology.orchestra.run.smithi155.stdout:Total 686 kB/s | 145 kB 00:00 2024-09-19T06:07:41.174 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-09-19T06:07:41.181 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-09-19T06:07:41.181 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-09-19T06:07:41.238 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-09-19T06:07:41.238 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-09-19T06:07:41.407 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-09-19T06:07:41.490 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.e 1/1 2024-09-19T06:07:41.515 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.e 1/1 2024-09-19T06:07:41.515 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:07:41.515 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T06:07:41.515 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:07:42.044 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noar 12/12 2024-09-19T06:07:42.044 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9 1/12 2024-09-19T06:07:42.044 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noar 2/12 2024-09-19T06:07:42.044 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9. 3/12 2024-09-19T06:07:42.044 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-19T06:07:42.044 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-19T06:07:42.044 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-19T06:07:42.044 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-19T06:07:42.044 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-19T06:07:42.044 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-19T06:07:42.044 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-19T06:07:42.045 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-19T06:07:42.411 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.e 1/1 2024-09-19T06:07:42.411 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:07:42.411 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-09-19T06:07:42.411 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:07:42.412 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:07:42.412 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-09-19T06:07:42.495 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-19T06:07:42.495 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:07:42.495 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-19T06:07:42.495 INFO:teuthology.orchestra.run.smithi053.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T06:07:42.496 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:07:42.496 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:07:42.496 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:07:42.496 INFO:teuthology.orchestra.run.smithi053.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T06:07:42.496 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T06:07:42.496 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T06:07:42.496 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T06:07:42.496 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T06:07:42.496 INFO:teuthology.orchestra.run.smithi053.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T06:07:42.496 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T06:07:42.496 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T06:07:42.496 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:07:42.497 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-19T06:07:42.660 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-mgr 2024-09-19T06:07:42.869 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-19T06:07:43.257 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:00:58 ago on Thu 19 Sep 2024 06:06:45 AM UTC. 2024-09-19T06:07:43.347 INFO:teuthology.orchestra.run.smithi155.stdout:Package ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 is already installed. 2024-09-19T06:07:43.389 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-09-19T06:07:43.390 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-09-19T06:07:43.390 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-09-19T06:07:43.453 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-mgr-dashboard 2024-09-19T06:07:43.478 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:09 ago on Thu 19 Sep 2024 06:06:34 AM UTC. 2024-09-19T06:07:43.618 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-19T06:07:43.619 INFO:teuthology.orchestra.run.smithi053.stdout:====================================================================================== 2024-09-19T06:07:43.619 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-19T06:07:43.619 INFO:teuthology.orchestra.run.smithi053.stdout:====================================================================================== 2024-09-19T06:07:43.619 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-19T06:07:43.619 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T06:07:43.620 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-19T06:07:43.620 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T06:07:43.620 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T06:07:43.620 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T06:07:43.620 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T06:07:43.620 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T06:07:43.620 INFO:teuthology.orchestra.run.smithi053.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T06:07:43.620 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T06:07:43.620 INFO:teuthology.orchestra.run.smithi053.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T06:07:43.620 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T06:07:43.620 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T06:07:43.620 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T06:07:43.620 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:07:43.621 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-19T06:07:43.621 INFO:teuthology.orchestra.run.smithi053.stdout:====================================================================================== 2024-09-19T06:07:43.621 INFO:teuthology.orchestra.run.smithi053.stdout:Install 12 Packages 2024-09-19T06:07:43.621 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:07:43.621 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 38 M 2024-09-19T06:07:43.621 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 212 M 2024-09-19T06:07:43.622 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-19T06:07:44.029 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:00:59 ago on Thu 19 Sep 2024 06:06:45 AM UTC. 2024-09-19T06:07:44.037 INFO:teuthology.orchestra.run.smithi053.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 191 kB 00:00 2024-09-19T06:07:44.096 INFO:teuthology.orchestra.run.smithi053.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.8 MB/s | 801 kB 00:00 2024-09-19T06:07:44.160 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-09-19T06:07:44.163 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:07:44.163 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-09-19T06:07:44.163 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:07:44.163 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-09-19T06:07:44.163 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T06:07:44.164 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-09-19T06:07:44.164 INFO:teuthology.orchestra.run.smithi155.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T06:07:44.164 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T06:07:44.164 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T06:07:44.164 INFO:teuthology.orchestra.run.smithi155.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T06:07:44.164 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T06:07:44.164 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T06:07:44.164 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T06:07:44.164 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T06:07:44.164 INFO:teuthology.orchestra.run.smithi155.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T06:07:44.164 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T06:07:44.164 INFO:teuthology.orchestra.run.smithi155.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T06:07:44.164 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:07:44.165 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-09-19T06:07:44.165 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:07:44.165 INFO:teuthology.orchestra.run.smithi155.stdout:Install 12 Packages 2024-09-19T06:07:44.165 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:07:44.165 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 8.7 M 2024-09-19T06:07:44.165 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 105 M 2024-09-19T06:07:44.165 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-09-19T06:07:44.321 INFO:teuthology.orchestra.run.smithi053.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 115 kB/s | 33 kB 00:00 2024-09-19T06:07:44.413 INFO:teuthology.orchestra.run.smithi053.stdout:(4/12): ceph-mgr-diskprediction-local-19.1.1-36 14 MB/s | 7.4 MB 00:00 2024-09-19T06:07:44.438 INFO:teuthology.orchestra.run.smithi053.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 156 kB/s | 18 kB 00:00 2024-09-19T06:07:44.577 INFO:teuthology.orchestra.run.smithi155.stdout:(1/12): ceph-grafana-dashboards-19.1.1-361.g2f5 208 kB/s | 32 kB 00:00 2024-09-19T06:07:44.602 INFO:teuthology.orchestra.run.smithi155.stdout:(2/12): ceph-prometheus-alerts-19.1.1-361.g2f59 93 kB/s | 16 kB 00:00 2024-09-19T06:07:44.705 INFO:teuthology.orchestra.run.smithi053.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 132 kB/s | 38 kB 00:00 2024-09-19T06:07:44.744 INFO:teuthology.orchestra.run.smithi155.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.9 MB/s | 269 kB 00:00 2024-09-19T06:07:44.795 INFO:teuthology.orchestra.run.smithi155.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.6 MB/s | 1.0 MB 00:00 2024-09-19T06:07:44.870 INFO:teuthology.orchestra.run.smithi155.stdout:(5/12): ceph-mgr-dashboard-19.1.1-361.g2f599a46 8.0 MB/s | 3.6 MB 00:00 2024-09-19T06:07:44.929 INFO:teuthology.orchestra.run.smithi155.stdout:(6/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 4.0 MB/s | 548 kB 00:00 2024-09-19T06:07:44.954 INFO:teuthology.orchestra.run.smithi155.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 237 kB/s | 19 kB 00:00 2024-09-19T06:07:44.979 INFO:teuthology.orchestra.run.smithi155.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.7 MB/s | 144 kB 00:00 2024-09-19T06:07:45.004 INFO:teuthology.orchestra.run.smithi155.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-19T06:07:45.030 INFO:teuthology.orchestra.run.smithi155.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 7.3 MB/s | 188 kB 00:00 2024-09-19T06:07:45.080 INFO:teuthology.orchestra.run.smithi155.stdout:(11/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 14 MB/s | 2.0 MB 00:00 2024-09-19T06:07:45.114 INFO:teuthology.orchestra.run.smithi155.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 2.3 MB/s | 863 kB 00:00 2024-09-19T06:07:45.117 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:07:45.117 INFO:teuthology.orchestra.run.smithi155.stdout:Total 9.1 MB/s | 8.7 MB 00:00 2024-09-19T06:07:45.165 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-09-19T06:07:45.181 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-09-19T06:07:45.181 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-09-19T06:07:45.189 INFO:teuthology.orchestra.run.smithi053.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 506 kB/s | 245 kB 00:00 2024-09-19T06:07:45.289 INFO:teuthology.orchestra.run.smithi053.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.5 MB/s | 3.0 MB 00:01 2024-09-19T06:07:45.406 INFO:teuthology.orchestra.run.smithi053.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 5.1 MB/s | 4.9 MB 00:00 2024-09-19T06:07:45.410 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-09-19T06:07:45.410 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-09-19T06:07:45.431 INFO:teuthology.orchestra.run.smithi053.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 1.7 MB/s | 250 kB 00:00 2024-09-19T06:07:45.946 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-09-19T06:07:45.948 INFO:teuthology.orchestra.run.smithi053.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.7 MB/s | 5.1 MB 00:00 2024-09-19T06:07:46.031 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-19T06:07:46.074 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-19T06:07:46.121 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-19T06:07:46.151 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-19T06:07:46.262 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-19T06:07:46.386 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-19T06:07:46.480 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-19T06:07:46.532 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-19T06:07:46.561 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-19T06:07:46.592 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9. 10/12 2024-09-19T06:07:47.650 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9 11/12 2024-09-19T06:07:47.693 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noar 12/12 2024-09-19T06:07:49.102 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noar 12/12 2024-09-19T06:07:49.102 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9 1/12 2024-09-19T06:07:49.102 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noar 2/12 2024-09-19T06:07:49.102 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9. 3/12 2024-09-19T06:07:49.102 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-19T06:07:49.103 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-19T06:07:49.103 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-19T06:07:49.103 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-19T06:07:49.103 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-19T06:07:49.103 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-19T06:07:49.103 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-19T06:07:49.103 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-19T06:07:49.452 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-19T06:07:49.452 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:07:49.452 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-09-19T06:07:49.452 INFO:teuthology.orchestra.run.smithi155.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T06:07:49.452 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:07:49.452 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:07:49.452 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:07:49.453 INFO:teuthology.orchestra.run.smithi155.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T06:07:49.453 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T06:07:49.453 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T06:07:49.453 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T06:07:49.453 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T06:07:49.453 INFO:teuthology.orchestra.run.smithi155.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T06:07:49.453 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T06:07:49.453 INFO:teuthology.orchestra.run.smithi155.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T06:07:49.453 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:07:49.453 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-09-19T06:07:49.734 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-19T06:07:50.311 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:05 ago on Thu 19 Sep 2024 06:06:45 AM UTC. 2024-09-19T06:07:50.444 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-09-19T06:07:50.445 INFO:teuthology.orchestra.run.smithi155.stdout:====================================================================================== 2024-09-19T06:07:50.445 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-09-19T06:07:50.445 INFO:teuthology.orchestra.run.smithi155.stdout:====================================================================================== 2024-09-19T06:07:50.446 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-09-19T06:07:50.446 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T06:07:50.446 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-09-19T06:07:50.446 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T06:07:50.446 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T06:07:50.446 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T06:07:50.446 INFO:teuthology.orchestra.run.smithi155.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T06:07:50.446 INFO:teuthology.orchestra.run.smithi155.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T06:07:50.446 INFO:teuthology.orchestra.run.smithi155.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T06:07:50.446 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T06:07:50.446 INFO:teuthology.orchestra.run.smithi155.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T06:07:50.446 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T06:07:50.446 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T06:07:50.447 INFO:teuthology.orchestra.run.smithi155.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T06:07:50.447 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:07:50.447 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-09-19T06:07:50.447 INFO:teuthology.orchestra.run.smithi155.stdout:====================================================================================== 2024-09-19T06:07:50.447 INFO:teuthology.orchestra.run.smithi155.stdout:Install 12 Packages 2024-09-19T06:07:50.447 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:07:50.447 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 38 M 2024-09-19T06:07:50.448 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 212 M 2024-09-19T06:07:50.448 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-09-19T06:07:51.148 INFO:teuthology.orchestra.run.smithi155.stdout:(1/12): ceph-mgr-diskprediction-local-19.1.1-36 16 MB/s | 7.4 MB 00:00 2024-09-19T06:07:51.214 INFO:teuthology.orchestra.run.smithi155.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 492 kB/s | 33 kB 00:00 2024-09-19T06:07:51.390 INFO:teuthology.orchestra.run.smithi155.stdout:(3/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 17 MB/s | 3.0 MB 00:00 2024-09-19T06:07:51.432 INFO:teuthology.orchestra.run.smithi155.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 439 kB/s | 18 kB 00:00 2024-09-19T06:07:51.474 INFO:teuthology.orchestra.run.smithi155.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 920 kB/s | 38 kB 00:00 2024-09-19T06:07:51.516 INFO:teuthology.orchestra.run.smithi155.stdout:(6/12): libquadmath-11.5.0-2.el9.x86_64.rpm 229 kB/s | 191 kB 00:00 2024-09-19T06:07:51.625 INFO:teuthology.orchestra.run.smithi155.stdout:(7/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 33 MB/s | 4.9 MB 00:00 2024-09-19T06:07:51.708 INFO:teuthology.orchestra.run.smithi155.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.2 MB/s | 245 kB 00:00 2024-09-19T06:07:51.775 INFO:teuthology.orchestra.run.smithi155.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 5.1 MB 00:00 2024-09-19T06:07:51.809 INFO:teuthology.orchestra.run.smithi155.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.4 MB/s | 250 kB 00:00 2024-09-19T06:07:52.109 INFO:teuthology.orchestra.run.smithi155.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 49 MB/s | 16 MB 00:00 2024-09-19T06:07:53.326 INFO:teuthology.orchestra.run.smithi155.stdout:(12/12): libgfortran-11.5.0-2.el9.x86_64.rpm 303 kB/s | 801 kB 00:02 2024-09-19T06:07:53.326 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:07:53.327 INFO:teuthology.orchestra.run.smithi155.stdout:Total 13 MB/s | 38 MB 00:02 2024-09-19T06:07:53.684 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-09-19T06:07:53.711 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-09-19T06:07:53.711 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-09-19T06:07:54.154 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-09-19T06:07:54.155 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-09-19T06:07:54.926 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-09-19T06:07:55.005 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-19T06:07:55.049 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-19T06:07:55.149 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-19T06:07:55.201 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-19T06:07:55.387 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-19T06:07:55.424 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-19T06:07:55.501 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-19T06:07:56.170 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-19T06:07:56.280 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-19T06:07:57.561 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-19T06:07:57.732 INFO:teuthology.orchestra.run.smithi053.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 1.3 MB/s | 16 MB 00:12 2024-09-19T06:07:57.732 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:07:57.733 INFO:teuthology.orchestra.run.smithi053.stdout:Total 2.7 MB/s | 38 MB 00:14 2024-09-19T06:07:58.102 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-19T06:07:58.129 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-19T06:07:58.130 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-19T06:07:58.585 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-19T06:07:58.586 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-19T06:07:58.641 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-19T06:07:58.667 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a 12/12 2024-09-19T06:07:59.491 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-19T06:07:59.563 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-19T06:07:59.605 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-19T06:07:59.708 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-19T06:07:59.756 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-19T06:07:59.948 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-19T06:07:59.979 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-19T06:08:00.058 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-19T06:08:00.751 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-19T06:08:00.871 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-19T06:08:01.357 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a 12/12 2024-09-19T06:08:01.358 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a 1/12 2024-09-19T06:08:01.358 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-19T06:08:01.358 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-19T06:08:01.358 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-19T06:08:01.358 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-19T06:08:01.358 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-19T06:08:01.358 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-19T06:08:01.358 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-19T06:08:01.358 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-19T06:08:01.358 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-19T06:08:01.358 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-19T06:08:01.812 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-19T06:08:01.812 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:01.812 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-09-19T06:08:01.812 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:08:01.812 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T06:08:01.812 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T06:08:01.812 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T06:08:01.812 INFO:teuthology.orchestra.run.smithi155.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T06:08:01.812 INFO:teuthology.orchestra.run.smithi155.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T06:08:01.813 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T06:08:01.813 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T06:08:01.813 INFO:teuthology.orchestra.run.smithi155.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T06:08:01.813 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T06:08:01.813 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T06:08:01.813 INFO:teuthology.orchestra.run.smithi155.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T06:08:01.813 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:01.813 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-09-19T06:08:02.186 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-19T06:08:02.295 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-mgr-rook 2024-09-19T06:08:02.887 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:17 ago on Thu 19 Sep 2024 06:06:45 AM UTC. 2024-09-19T06:08:03.021 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-09-19T06:08:03.022 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:08:03.022 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-09-19T06:08:03.022 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:08:03.022 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-09-19T06:08:03.022 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T06:08:03.022 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-09-19T06:08:03.022 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T06:08:03.022 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T06:08:03.022 INFO:teuthology.orchestra.run.smithi155.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T06:08:03.023 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T06:08:03.023 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T06:08:03.023 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T06:08:03.023 INFO:teuthology.orchestra.run.smithi155.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T06:08:03.023 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T06:08:03.023 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T06:08:03.023 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T06:08:03.023 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T06:08:03.023 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T06:08:03.023 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:03.023 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-09-19T06:08:03.023 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:08:03.023 INFO:teuthology.orchestra.run.smithi155.stdout:Install 13 Packages 2024-09-19T06:08:03.023 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:03.024 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 2.2 M 2024-09-19T06:08:03.024 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 26 M 2024-09-19T06:08:03.024 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-09-19T06:08:03.279 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-19T06:08:03.311 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a 12/12 2024-09-19T06:08:03.636 INFO:teuthology.orchestra.run.smithi155.stdout:(1/13): ceph-mgr-rook-19.1.1-361.g2f599a46.el9. 303 kB/s | 49 kB 00:00 2024-09-19T06:08:04.903 INFO:teuthology.orchestra.run.smithi155.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 13 kB/s | 19 kB 00:01 2024-09-19T06:08:04.928 INFO:teuthology.orchestra.run.smithi155.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 18 kB/s | 26 kB 00:01 2024-09-19T06:08:05.012 INFO:teuthology.orchestra.run.smithi155.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 161 kB/s | 222 kB 00:01 2024-09-19T06:08:05.254 INFO:teuthology.orchestra.run.smithi155.stdout:(5/13): python3-requests-oauthlib-1.3.0-12.el9. 222 kB/s | 54 kB 00:00 2024-09-19T06:08:05.287 INFO:teuthology.orchestra.run.smithi155.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 971 kB/s | 32 kB 00:00 2024-09-19T06:08:05.304 INFO:teuthology.orchestra.run.smithi155.stdout:(7/13): python3-certifi-2023.05.07-4.el9.noarch 847 kB/s | 14 kB 00:00 2024-09-19T06:08:05.346 INFO:teuthology.orchestra.run.smithi155.stdout:(8/13): python3-google-auth-2.34.0-1.el9.noarch 5.2 MB/s | 220 kB 00:00 2024-09-19T06:08:05.371 INFO:teuthology.orchestra.run.smithi155.stdout:(9/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 339 kB/s | 159 kB 00:00 2024-09-19T06:08:05.405 INFO:teuthology.orchestra.run.smithi155.stdout:(10/13): python3-pyasn1-modules-0.4.8-6.el9.noa 585 kB/s | 279 kB 00:00 2024-09-19T06:08:05.439 INFO:teuthology.orchestra.run.smithi155.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 883 kB/s | 59 kB 00:00 2024-09-19T06:08:05.464 INFO:teuthology.orchestra.run.smithi155.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.5 MB/s | 90 kB 00:00 2024-09-19T06:08:05.498 INFO:teuthology.orchestra.run.smithi155.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.8 MB/s | 1.0 MB 00:00 2024-09-19T06:08:05.500 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:08:05.501 INFO:teuthology.orchestra.run.smithi155.stdout:Total 915 kB/s | 2.2 MB 00:02 2024-09-19T06:08:05.571 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-09-19T06:08:05.587 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-09-19T06:08:05.587 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-09-19T06:08:05.682 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-09-19T06:08:05.684 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-09-19T06:08:05.710 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a 12/12 2024-09-19T06:08:05.711 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a 1/12 2024-09-19T06:08:05.711 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-19T06:08:05.711 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-19T06:08:05.711 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-19T06:08:05.711 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-19T06:08:05.711 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-19T06:08:05.711 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-19T06:08:05.711 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-19T06:08:05.711 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-19T06:08:05.711 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-19T06:08:05.711 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-19T06:08:06.134 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-19T06:08:06.134 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:06.135 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-19T06:08:06.135 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:08:06.135 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T06:08:06.135 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T06:08:06.135 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T06:08:06.135 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T06:08:06.135 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T06:08:06.135 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T06:08:06.135 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T06:08:06.135 INFO:teuthology.orchestra.run.smithi053.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T06:08:06.135 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T06:08:06.135 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T06:08:06.136 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T06:08:06.136 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:06.136 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-19T06:08:06.221 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-09-19T06:08:06.387 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-19T06:08:06.450 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-19T06:08:06.504 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-19T06:08:06.550 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-19T06:08:06.588 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-mgr-rook 2024-09-19T06:08:06.593 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-19T06:08:06.694 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-19T06:08:06.804 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-19T06:08:06.859 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-19T06:08:07.192 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:33 ago on Thu 19 Sep 2024 06:06:34 AM UTC. 2024-09-19T06:08:07.331 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-19T06:08:07.333 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:08:07.333 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-19T06:08:07.333 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:08:07.333 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-19T06:08:07.333 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T06:08:07.333 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-19T06:08:07.333 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T06:08:07.333 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T06:08:07.333 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T06:08:07.333 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T06:08:07.333 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T06:08:07.333 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T06:08:07.334 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T06:08:07.334 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T06:08:07.334 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T06:08:07.334 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T06:08:07.334 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T06:08:07.334 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T06:08:07.334 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:07.334 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-19T06:08:07.334 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:08:07.334 INFO:teuthology.orchestra.run.smithi053.stdout:Install 13 Packages 2024-09-19T06:08:07.334 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:07.335 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 2.2 M 2024-09-19T06:08:07.335 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 26 M 2024-09-19T06:08:07.335 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-19T06:08:07.544 INFO:teuthology.orchestra.run.smithi053.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 217 kB/s | 26 kB 00:00 2024-09-19T06:08:07.561 INFO:teuthology.orchestra.run.smithi053.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 142 kB/s | 19 kB 00:00 2024-09-19T06:08:07.578 INFO:teuthology.orchestra.run.smithi053.stdout:(3/13): ceph-mgr-rook-19.1.1-361.g2f599a46.el9. 324 kB/s | 49 kB 00:00 2024-09-19T06:08:07.583 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-19T06:08:07.633 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-19T06:08:07.666 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-19T06:08:07.703 INFO:teuthology.orchestra.run.smithi053.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-09-19T06:08:07.714 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-19T06:08:07.728 INFO:teuthology.orchestra.run.smithi053.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.2 MB/s | 222 kB 00:00 2024-09-19T06:08:07.736 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 13/13 2024-09-19T06:08:07.745 INFO:teuthology.orchestra.run.smithi053.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.2 MB/s | 54 kB 00:00 2024-09-19T06:08:07.779 INFO:teuthology.orchestra.run.smithi053.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.4 MB/s | 279 kB 00:00 2024-09-19T06:08:07.812 INFO:teuthology.orchestra.run.smithi053.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 383 kB/s | 32 kB 00:00 2024-09-19T06:08:07.846 INFO:teuthology.orchestra.run.smithi053.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 151 kB/s | 14 kB 00:00 2024-09-19T06:08:07.879 INFO:teuthology.orchestra.run.smithi053.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.8 MB/s | 59 kB 00:00 2024-09-19T06:08:07.922 INFO:teuthology.orchestra.run.smithi053.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 2.1 MB/s | 90 kB 00:00 2024-09-19T06:08:07.947 INFO:teuthology.orchestra.run.smithi053.stdout:(12/13): python3-google-auth-2.34.0-1.el9.noarc 1.4 MB/s | 220 kB 00:00 2024-09-19T06:08:07.980 INFO:teuthology.orchestra.run.smithi053.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.4 MB/s | 1.0 MB 00:00 2024-09-19T06:08:07.983 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:08:07.983 INFO:teuthology.orchestra.run.smithi053.stdout:Total 3.4 MB/s | 2.2 MB 00:00 2024-09-19T06:08:08.050 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-19T06:08:08.066 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-19T06:08:08.067 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-19T06:08:08.168 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-19T06:08:08.169 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-19T06:08:08.527 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 13/13 2024-09-19T06:08:08.527 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 1/13 2024-09-19T06:08:08.528 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-19T06:08:08.528 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-19T06:08:08.528 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-19T06:08:08.528 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-19T06:08:08.528 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-19T06:08:08.528 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-19T06:08:08.528 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-19T06:08:08.528 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-19T06:08:08.528 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-19T06:08:08.528 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-19T06:08:08.528 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-19T06:08:08.700 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-19T06:08:08.865 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-19T06:08:08.866 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:08.866 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-09-19T06:08:08.866 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:08:08.866 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T06:08:08.866 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T06:08:08.866 INFO:teuthology.orchestra.run.smithi155.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T06:08:08.866 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T06:08:08.866 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T06:08:08.866 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T06:08:08.866 INFO:teuthology.orchestra.run.smithi155.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T06:08:08.866 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T06:08:08.867 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T06:08:08.867 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T06:08:08.867 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T06:08:08.867 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T06:08:08.867 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:08.867 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-09-19T06:08:08.877 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-19T06:08:08.941 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-19T06:08:08.998 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-19T06:08:09.032 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-19T06:08:09.101 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-19T06:08:09.125 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-mgr-cephadm 2024-09-19T06:08:09.207 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-19T06:08:09.325 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-19T06:08:09.385 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-19T06:08:09.709 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:24 ago on Thu 19 Sep 2024 06:06:45 AM UTC. 2024-09-19T06:08:09.843 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-09-19T06:08:09.844 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:08:09.844 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-09-19T06:08:09.844 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:08:09.844 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-09-19T06:08:09.844 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T06:08:09.844 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-09-19T06:08:09.844 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T06:08:09.844 INFO:teuthology.orchestra.run.smithi155.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T06:08:09.845 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T06:08:09.845 INFO:teuthology.orchestra.run.smithi155.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T06:08:09.845 INFO:teuthology.orchestra.run.smithi155.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T06:08:09.845 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:09.845 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-09-19T06:08:09.845 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:08:09.845 INFO:teuthology.orchestra.run.smithi155.stdout:Install 6 Packages 2024-09-19T06:08:09.845 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:09.846 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 7.0 M 2024-09-19T06:08:09.846 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 33 M 2024-09-19T06:08:09.846 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-09-19T06:08:10.145 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-19T06:08:10.198 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-19T06:08:10.232 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-19T06:08:10.281 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-19T06:08:10.326 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 13/13 2024-09-19T06:08:10.468 INFO:teuthology.orchestra.run.smithi155.stdout:(1/6): ceph-mgr-cephadm-19.1.1-361.g2f599a46.el 711 kB/s | 146 kB 00:00 2024-09-19T06:08:10.527 INFO:teuthology.orchestra.run.smithi155.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 9.2 MB/s | 548 kB 00:00 2024-09-19T06:08:10.552 INFO:teuthology.orchestra.run.smithi155.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-09-19T06:08:10.577 INFO:teuthology.orchestra.run.smithi155.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-19T06:08:11.027 INFO:teuthology.orchestra.run.smithi155.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 325 kB/s | 248 kB 00:00 2024-09-19T06:08:11.070 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 13/13 2024-09-19T06:08:11.070 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 1/13 2024-09-19T06:08:11.070 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-19T06:08:11.070 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-19T06:08:11.070 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-19T06:08:11.070 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-19T06:08:11.070 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-19T06:08:11.070 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-19T06:08:11.071 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-19T06:08:11.071 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-19T06:08:11.071 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-19T06:08:11.071 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-19T06:08:11.071 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-19T06:08:11.511 INFO:teuthology.orchestra.run.smithi155.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.8 MB/s | 6.0 MB 00:01 2024-09-19T06:08:11.513 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:08:11.513 INFO:teuthology.orchestra.run.smithi155.stdout:Total 4.2 MB/s | 7.0 MB 00:01 2024-09-19T06:08:11.532 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-19T06:08:11.532 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:11.532 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-19T06:08:11.532 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:08:11.532 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T06:08:11.532 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T06:08:11.532 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T06:08:11.532 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T06:08:11.532 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T06:08:11.533 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T06:08:11.533 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T06:08:11.533 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T06:08:11.533 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T06:08:11.533 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T06:08:11.533 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T06:08:11.533 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T06:08:11.533 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:11.533 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-19T06:08:11.583 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-09-19T06:08:11.594 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-09-19T06:08:11.594 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-09-19T06:08:11.693 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-09-19T06:08:11.693 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-09-19T06:08:11.821 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-mgr-cephadm 2024-09-19T06:08:11.995 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-09-19T06:08:12.168 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-19T06:08:12.221 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-19T06:08:12.431 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:38 ago on Thu 19 Sep 2024 06:06:34 AM UTC. 2024-09-19T06:08:12.568 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-19T06:08:12.569 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:08:12.569 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-19T06:08:12.569 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:08:12.569 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-19T06:08:12.569 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T06:08:12.569 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-19T06:08:12.569 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T06:08:12.569 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T06:08:12.570 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T06:08:12.570 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T06:08:12.570 INFO:teuthology.orchestra.run.smithi053.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T06:08:12.570 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:12.570 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-19T06:08:12.570 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:08:12.570 INFO:teuthology.orchestra.run.smithi053.stdout:Install 6 Packages 2024-09-19T06:08:12.570 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:12.571 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 7.0 M 2024-09-19T06:08:12.571 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 33 M 2024-09-19T06:08:12.571 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-19T06:08:12.753 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-19T06:08:12.840 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-19T06:08:12.904 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-19T06:08:12.933 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 6/6 2024-09-19T06:08:13.215 INFO:teuthology.orchestra.run.smithi053.stdout:(1/6): ceph-mgr-cephadm-19.1.1-361.g2f599a46.el 744 kB/s | 146 kB 00:00 2024-09-19T06:08:13.273 INFO:teuthology.orchestra.run.smithi053.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 9.2 MB/s | 548 kB 00:00 2024-09-19T06:08:13.299 INFO:teuthology.orchestra.run.smithi053.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-09-19T06:08:13.316 INFO:teuthology.orchestra.run.smithi053.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 4.4 MB/s | 74 kB 00:00 2024-09-19T06:08:13.568 INFO:teuthology.orchestra.run.smithi053.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 453 kB/s | 248 kB 00:00 2024-09-19T06:08:13.719 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 6/6 2024-09-19T06:08:13.719 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 1/6 2024-09-19T06:08:13.719 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-19T06:08:13.719 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-19T06:08:13.719 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-19T06:08:13.719 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-19T06:08:13.989 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-19T06:08:13.989 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:13.989 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-09-19T06:08:13.990 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:08:13.990 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T06:08:13.990 INFO:teuthology.orchestra.run.smithi155.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T06:08:13.990 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T06:08:13.990 INFO:teuthology.orchestra.run.smithi155.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T06:08:13.990 INFO:teuthology.orchestra.run.smithi155.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T06:08:13.990 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:13.990 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-09-19T06:08:14.269 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-fuse 2024-09-19T06:08:14.851 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:29 ago on Thu 19 Sep 2024 06:06:45 AM UTC. 2024-09-19T06:08:14.983 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-09-19T06:08:14.984 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:08:14.984 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-09-19T06:08:14.984 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:08:14.984 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-09-19T06:08:14.984 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T06:08:14.984 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-09-19T06:08:14.984 INFO:teuthology.orchestra.run.smithi155.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T06:08:14.984 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:14.984 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-09-19T06:08:14.984 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:08:14.985 INFO:teuthology.orchestra.run.smithi155.stdout:Install 2 Packages 2024-09-19T06:08:14.985 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:14.985 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 976 k 2024-09-19T06:08:14.985 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 2.9 M 2024-09-19T06:08:14.985 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-09-19T06:08:15.007 INFO:teuthology.orchestra.run.smithi053.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 3.0 MB/s | 6.0 MB 00:01 2024-09-19T06:08:15.009 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:08:15.010 INFO:teuthology.orchestra.run.smithi053.stdout:Total 2.9 MB/s | 7.0 MB 00:02 2024-09-19T06:08:15.086 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-19T06:08:15.099 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-19T06:08:15.099 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-19T06:08:15.114 INFO:teuthology.orchestra.run.smithi155.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 790 kB/s | 80 kB 00:00 2024-09-19T06:08:15.205 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-19T06:08:15.211 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-19T06:08:15.297 INFO:teuthology.orchestra.run.smithi155.stdout:(2/2): ceph-fuse-19.1.1-361.g2f599a46.el9.x86_6 3.1 MB/s | 896 kB 00:00 2024-09-19T06:08:15.298 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:08:15.298 INFO:teuthology.orchestra.run.smithi155.stdout:Total 3.0 MB/s | 976 kB 00:00 2024-09-19T06:08:15.310 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-09-19T06:08:15.331 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-09-19T06:08:15.331 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-09-19T06:08:15.412 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-09-19T06:08:15.413 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-09-19T06:08:15.527 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-19T06:08:15.605 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-09-19T06:08:15.705 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-19T06:08:15.767 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-19T06:08:15.768 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-19T06:08:15.839 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2/2 2024-09-19T06:08:16.321 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-19T06:08:16.406 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-19T06:08:16.485 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2/2 2024-09-19T06:08:16.485 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 1/2 2024-09-19T06:08:16.486 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-19T06:08:16.513 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 6/6 2024-09-19T06:08:16.774 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-19T06:08:16.774 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:16.774 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-09-19T06:08:16.775 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-19T06:08:16.775 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:16.775 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-09-19T06:08:17.053 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-volume 2024-09-19T06:08:17.259 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 6/6 2024-09-19T06:08:17.259 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 1/6 2024-09-19T06:08:17.259 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-19T06:08:17.259 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-19T06:08:17.259 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-19T06:08:17.259 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-19T06:08:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-19T06:08:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:17.540 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-19T06:08:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:08:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T06:08:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T06:08:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T06:08:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T06:08:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T06:08:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:17.542 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-19T06:08:17.651 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:32 ago on Thu 19 Sep 2024 06:06:45 AM UTC. 2024-09-19T06:08:17.788 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-09-19T06:08:17.789 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:08:17.789 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-09-19T06:08:17.789 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:08:17.789 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-09-19T06:08:17.789 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T06:08:17.789 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:17.789 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-09-19T06:08:17.789 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:08:17.789 INFO:teuthology.orchestra.run.smithi155.stdout:Install 1 Package 2024-09-19T06:08:17.789 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:17.790 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 285 k 2024-09-19T06:08:17.790 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 1.3 M 2024-09-19T06:08:17.790 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-09-19T06:08:17.799 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-fuse 2024-09-19T06:08:18.020 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-volume-19.1.1-361.g2f599a46.el9.noarch.rpm 1.2 MB/s | 285 kB 00:00 2024-09-19T06:08:18.020 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:08:18.021 INFO:teuthology.orchestra.run.smithi155.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-19T06:08:18.021 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-09-19T06:08:18.029 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-09-19T06:08:18.029 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-09-19T06:08:18.062 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-09-19T06:08:18.062 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-09-19T06:08:18.289 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-09-19T06:08:18.364 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 1/1 2024-09-19T06:08:18.379 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 1/1 2024-09-19T06:08:18.380 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:08:18.380 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T06:08:18.380 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:18.403 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:44 ago on Thu 19 Sep 2024 06:06:34 AM UTC. 2024-09-19T06:08:18.542 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-19T06:08:18.542 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:08:18.542 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-19T06:08:18.542 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:08:18.542 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-19T06:08:18.542 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T06:08:18.543 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-19T06:08:18.543 INFO:teuthology.orchestra.run.smithi053.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T06:08:18.543 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:18.543 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-19T06:08:18.543 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:08:18.543 INFO:teuthology.orchestra.run.smithi053.stdout:Install 2 Packages 2024-09-19T06:08:18.543 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:18.543 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 976 k 2024-09-19T06:08:18.544 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 2.9 M 2024-09-19T06:08:18.544 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-19T06:08:18.968 INFO:teuthology.orchestra.run.smithi053.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 546 kB/s | 80 kB 00:00 2024-09-19T06:08:19.093 INFO:teuthology.orchestra.run.smithi053.stdout:(2/2): ceph-fuse-19.1.1-361.g2f599a46.el9.x86_6 3.2 MB/s | 896 kB 00:00 2024-09-19T06:08:19.093 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:08:19.094 INFO:teuthology.orchestra.run.smithi053.stdout:Total 1.7 MB/s | 976 kB 00:00 2024-09-19T06:08:19.106 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-19T06:08:19.130 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-19T06:08:19.130 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-19T06:08:19.216 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-19T06:08:19.216 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-19T06:08:19.412 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-19T06:08:19.482 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 1/1 2024-09-19T06:08:19.482 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:19.483 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-09-19T06:08:19.483 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:08:19.483 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:19.483 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-09-19T06:08:19.549 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-19T06:08:19.628 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2/2 2024-09-19T06:08:19.809 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install librados-devel 2024-09-19T06:08:20.298 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2/2 2024-09-19T06:08:20.298 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 1/2 2024-09-19T06:08:20.410 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:35 ago on Thu 19 Sep 2024 06:06:45 AM UTC. 2024-09-19T06:08:20.542 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-09-19T06:08:20.543 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:08:20.543 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repo Size 2024-09-19T06:08:20.543 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:08:20.543 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-09-19T06:08:20.543 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T06:08:20.543 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:20.543 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-09-19T06:08:20.543 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:08:20.543 INFO:teuthology.orchestra.run.smithi155.stdout:Install 1 Package 2024-09-19T06:08:20.543 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:20.543 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 127 k 2024-09-19T06:08:20.544 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 456 k 2024-09-19T06:08:20.544 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-09-19T06:08:20.557 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-19T06:08:20.557 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:20.557 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-19T06:08:20.557 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-19T06:08:20.557 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:20.557 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-19T06:08:20.751 INFO:teuthology.orchestra.run.smithi155.stdout:librados-devel-19.1.1-361.g2f599a46.el9.x86_64. 611 kB/s | 127 kB 00:00 2024-09-19T06:08:20.752 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:08:20.752 INFO:teuthology.orchestra.run.smithi155.stdout:Total 606 kB/s | 127 kB 00:00 2024-09-19T06:08:20.752 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-09-19T06:08:20.757 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-09-19T06:08:20.758 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-09-19T06:08:20.821 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-09-19T06:08:20.822 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-09-19T06:08:20.832 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-volume 2024-09-19T06:08:20.971 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-09-19T06:08:21.092 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 1/1 2024-09-19T06:08:21.441 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:47 ago on Thu 19 Sep 2024 06:06:34 AM UTC. 2024-09-19T06:08:21.558 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 1/1 2024-09-19T06:08:21.581 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-19T06:08:21.588 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:08:21.588 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-19T06:08:21.588 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:08:21.588 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-19T06:08:21.588 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T06:08:21.588 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:21.588 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-19T06:08:21.588 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:08:21.588 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-09-19T06:08:21.589 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:21.589 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 285 k 2024-09-19T06:08:21.589 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 1.3 M 2024-09-19T06:08:21.589 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-19T06:08:21.822 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-volume-19.1.1-361.g2f599a46.el9.noarch.rpm 1.2 MB/s | 285 kB 00:00 2024-09-19T06:08:21.822 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:08:21.823 INFO:teuthology.orchestra.run.smithi053.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-19T06:08:21.823 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-19T06:08:21.823 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 1/1 2024-09-19T06:08:21.823 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:21.824 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-09-19T06:08:21.824 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:08:21.824 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:21.824 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-09-19T06:08:21.836 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-19T06:08:21.836 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-19T06:08:21.867 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-19T06:08:21.868 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-19T06:08:22.026 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install libcephfs2 2024-09-19T06:08:22.099 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-19T06:08:22.174 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 1/1 2024-09-19T06:08:22.192 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 1/1 2024-09-19T06:08:22.192 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:08:22.192 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T06:08:22.192 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:22.615 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:37 ago on Thu 19 Sep 2024 06:06:45 AM UTC. 2024-09-19T06:08:22.707 INFO:teuthology.orchestra.run.smithi155.stdout:Package libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 is already installed. 2024-09-19T06:08:22.752 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-09-19T06:08:22.753 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-09-19T06:08:22.753 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-09-19T06:08:22.818 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install libcephfs-devel 2024-09-19T06:08:23.201 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 1/1 2024-09-19T06:08:23.201 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:23.201 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-19T06:08:23.201 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:08:23.201 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:23.201 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-19T06:08:23.396 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:38 ago on Thu 19 Sep 2024 06:06:45 AM UTC. 2024-09-19T06:08:23.527 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-09-19T06:08:23.527 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:08:23.528 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repo Size 2024-09-19T06:08:23.528 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:08:23.528 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-09-19T06:08:23.528 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T06:08:23.528 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:23.528 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-09-19T06:08:23.528 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:08:23.528 INFO:teuthology.orchestra.run.smithi155.stdout:Install 1 Package 2024-09-19T06:08:23.528 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:23.528 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 32 k 2024-09-19T06:08:23.528 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 142 k 2024-09-19T06:08:23.529 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-09-19T06:08:23.530 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install librados-devel 2024-09-19T06:08:23.680 INFO:teuthology.orchestra.run.smithi155.stdout:libcephfs-devel-19.1.1-361.g2f599a46.el9.x86_64 209 kB/s | 32 kB 00:00 2024-09-19T06:08:23.681 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:08:23.681 INFO:teuthology.orchestra.run.smithi155.stdout:Total 207 kB/s | 32 kB 00:00 2024-09-19T06:08:23.681 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-09-19T06:08:23.684 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-09-19T06:08:23.684 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-09-19T06:08:23.701 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-09-19T06:08:23.701 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-09-19T06:08:23.794 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-09-19T06:08:23.910 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 1/1 2024-09-19T06:08:24.143 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:50 ago on Thu 19 Sep 2024 06:06:34 AM UTC. 2024-09-19T06:08:24.282 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-19T06:08:24.283 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:08:24.283 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repo Size 2024-09-19T06:08:24.283 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:08:24.284 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-19T06:08:24.284 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T06:08:24.284 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:24.284 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-19T06:08:24.284 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:08:24.284 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-09-19T06:08:24.284 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:24.284 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 127 k 2024-09-19T06:08:24.284 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 456 k 2024-09-19T06:08:24.284 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-19T06:08:24.309 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 1/1 2024-09-19T06:08:24.495 INFO:teuthology.orchestra.run.smithi053.stdout:librados-devel-19.1.1-361.g2f599a46.el9.x86_64. 602 kB/s | 127 kB 00:00 2024-09-19T06:08:24.495 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:08:24.496 INFO:teuthology.orchestra.run.smithi053.stdout:Total 597 kB/s | 127 kB 00:00 2024-09-19T06:08:24.496 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-19T06:08:24.502 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-19T06:08:24.502 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-19T06:08:24.545 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 1/1 2024-09-19T06:08:24.545 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:24.545 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-09-19T06:08:24.545 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:08:24.545 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:24.545 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-09-19T06:08:24.570 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-19T06:08:24.571 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-19T06:08:24.711 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install librados2 2024-09-19T06:08:24.734 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-19T06:08:24.837 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 1/1 2024-09-19T06:08:25.293 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:40 ago on Thu 19 Sep 2024 06:06:45 AM UTC. 2024-09-19T06:08:25.319 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 1/1 2024-09-19T06:08:25.384 INFO:teuthology.orchestra.run.smithi155.stdout:Package librados2-2:19.1.1-361.g2f599a46.el9.x86_64 is already installed. 2024-09-19T06:08:25.427 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-09-19T06:08:25.428 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-09-19T06:08:25.428 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-09-19T06:08:25.489 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install librbd1 2024-09-19T06:08:25.596 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 1/1 2024-09-19T06:08:25.596 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:25.596 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-19T06:08:25.597 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:08:25.597 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:25.597 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-19T06:08:25.839 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install libcephfs2 2024-09-19T06:08:26.074 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:41 ago on Thu 19 Sep 2024 06:06:45 AM UTC. 2024-09-19T06:08:26.164 INFO:teuthology.orchestra.run.smithi155.stdout:Package librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 is already installed. 2024-09-19T06:08:26.208 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-09-19T06:08:26.209 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-09-19T06:08:26.209 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-09-19T06:08:26.283 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install python3-rados 2024-09-19T06:08:26.453 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:52 ago on Thu 19 Sep 2024 06:06:34 AM UTC. 2024-09-19T06:08:26.546 INFO:teuthology.orchestra.run.smithi053.stdout:Package libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 is already installed. 2024-09-19T06:08:26.591 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-19T06:08:26.593 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-19T06:08:26.593 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-19T06:08:26.661 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install libcephfs-devel 2024-09-19T06:08:26.866 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:41 ago on Thu 19 Sep 2024 06:06:45 AM UTC. 2024-09-19T06:08:26.954 INFO:teuthology.orchestra.run.smithi155.stdout:Package python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 is already installed. 2024-09-19T06:08:26.997 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-09-19T06:08:26.998 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-09-19T06:08:26.998 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-09-19T06:08:27.062 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install python3-rgw 2024-09-19T06:08:27.267 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:53 ago on Thu 19 Sep 2024 06:06:34 AM UTC. 2024-09-19T06:08:27.407 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-19T06:08:27.407 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:08:27.407 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repo Size 2024-09-19T06:08:27.407 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:08:27.408 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-19T06:08:27.408 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T06:08:27.408 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:27.408 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-19T06:08:27.408 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:08:27.408 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-09-19T06:08:27.408 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:27.408 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 32 k 2024-09-19T06:08:27.408 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 142 k 2024-09-19T06:08:27.408 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-19T06:08:27.557 INFO:teuthology.orchestra.run.smithi053.stdout:libcephfs-devel-19.1.1-361.g2f599a46.el9.x86_64 214 kB/s | 32 kB 00:00 2024-09-19T06:08:27.557 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:08:27.557 INFO:teuthology.orchestra.run.smithi053.stdout:Total 212 kB/s | 32 kB 00:00 2024-09-19T06:08:27.558 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-19T06:08:27.561 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-19T06:08:27.561 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-19T06:08:27.573 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-19T06:08:27.574 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-19T06:08:27.645 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:42 ago on Thu 19 Sep 2024 06:06:45 AM UTC. 2024-09-19T06:08:27.668 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-19T06:08:27.735 INFO:teuthology.orchestra.run.smithi155.stdout:Package python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 is already installed. 2024-09-19T06:08:27.765 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 1/1 2024-09-19T06:08:27.778 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-09-19T06:08:27.779 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-09-19T06:08:27.779 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-09-19T06:08:27.850 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install python3-cephfs 2024-09-19T06:08:28.100 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 1/1 2024-09-19T06:08:28.345 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 1/1 2024-09-19T06:08:28.346 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:28.346 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-19T06:08:28.346 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:08:28.346 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:28.346 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-19T06:08:28.428 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:43 ago on Thu 19 Sep 2024 06:06:45 AM UTC. 2024-09-19T06:08:28.518 INFO:teuthology.orchestra.run.smithi155.stdout:Package python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 is already installed. 2024-09-19T06:08:28.545 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install librados2 2024-09-19T06:08:28.561 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-09-19T06:08:28.562 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-09-19T06:08:28.562 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-09-19T06:08:28.630 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install python3-rbd 2024-09-19T06:08:29.146 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:55 ago on Thu 19 Sep 2024 06:06:34 AM UTC. 2024-09-19T06:08:29.237 INFO:teuthology.orchestra.run.smithi053.stdout:Package librados2-2:19.1.1-361.g2f599a46.el9.x86_64 is already installed. 2024-09-19T06:08:29.238 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:44 ago on Thu 19 Sep 2024 06:06:45 AM UTC. 2024-09-19T06:08:29.284 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-19T06:08:29.285 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-19T06:08:29.285 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-19T06:08:29.329 INFO:teuthology.orchestra.run.smithi155.stdout:Package python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 is already installed. 2024-09-19T06:08:29.356 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install librbd1 2024-09-19T06:08:29.375 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-09-19T06:08:29.376 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-09-19T06:08:29.376 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-09-19T06:08:29.439 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install rbd-fuse 2024-09-19T06:08:29.958 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:55 ago on Thu 19 Sep 2024 06:06:34 AM UTC. 2024-09-19T06:08:30.032 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:45 ago on Thu 19 Sep 2024 06:06:45 AM UTC. 2024-09-19T06:08:30.050 INFO:teuthology.orchestra.run.smithi053.stdout:Package librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 is already installed. 2024-09-19T06:08:30.097 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-19T06:08:30.098 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-19T06:08:30.098 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-19T06:08:30.164 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install python3-rados 2024-09-19T06:08:30.165 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-09-19T06:08:30.166 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:08:30.166 INFO:teuthology.orchestra.run.smithi155.stdout: Package Architecture Version Repository Size 2024-09-19T06:08:30.166 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:08:30.166 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-09-19T06:08:30.166 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T06:08:30.167 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:30.167 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-09-19T06:08:30.167 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:08:30.167 INFO:teuthology.orchestra.run.smithi155.stdout:Install 1 Package 2024-09-19T06:08:30.167 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:30.167 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 85 k 2024-09-19T06:08:30.167 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 231 k 2024-09-19T06:08:30.167 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-09-19T06:08:30.359 INFO:teuthology.orchestra.run.smithi155.stdout:rbd-fuse-19.1.1-361.g2f599a46.el9.x86_64.rpm 445 kB/s | 85 kB 00:00 2024-09-19T06:08:30.360 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:08:30.360 INFO:teuthology.orchestra.run.smithi155.stdout:Total 441 kB/s | 85 kB 00:00 2024-09-19T06:08:30.360 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-09-19T06:08:30.367 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-09-19T06:08:30.367 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-09-19T06:08:30.430 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-09-19T06:08:30.430 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-09-19T06:08:30.579 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-09-19T06:08:30.680 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 1/1 2024-09-19T06:08:30.776 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:56 ago on Thu 19 Sep 2024 06:06:34 AM UTC. 2024-09-19T06:08:30.869 INFO:teuthology.orchestra.run.smithi053.stdout:Package python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 is already installed. 2024-09-19T06:08:30.915 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-19T06:08:30.916 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-19T06:08:30.916 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-19T06:08:30.983 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install python3-rgw 2024-09-19T06:08:31.099 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 1/1 2024-09-19T06:08:31.362 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 1/1 2024-09-19T06:08:31.363 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:31.363 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-09-19T06:08:31.363 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:08:31.363 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:31.363 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-09-19T06:08:31.592 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:57 ago on Thu 19 Sep 2024 06:06:34 AM UTC. 2024-09-19T06:08:31.685 INFO:teuthology.orchestra.run.smithi053.stdout:Package python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 is already installed. 2024-09-19T06:08:31.692 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install rbd-mirror 2024-09-19T06:08:31.730 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-19T06:08:31.731 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-19T06:08:31.732 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-19T06:08:31.797 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install python3-cephfs 2024-09-19T06:08:32.289 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:47 ago on Thu 19 Sep 2024 06:06:45 AM UTC. 2024-09-19T06:08:32.400 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:58 ago on Thu 19 Sep 2024 06:06:34 AM UTC. 2024-09-19T06:08:32.423 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-09-19T06:08:32.424 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:08:32.424 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repo Size 2024-09-19T06:08:32.424 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:08:32.424 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-09-19T06:08:32.425 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T06:08:32.425 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:32.425 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-09-19T06:08:32.425 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:08:32.425 INFO:teuthology.orchestra.run.smithi155.stdout:Install 1 Package 2024-09-19T06:08:32.425 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:32.425 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 3.1 M 2024-09-19T06:08:32.425 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 13 M 2024-09-19T06:08:32.425 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-09-19T06:08:32.492 INFO:teuthology.orchestra.run.smithi053.stdout:Package python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 is already installed. 2024-09-19T06:08:32.539 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-19T06:08:32.540 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-19T06:08:32.540 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-19T06:08:32.614 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install python3-rbd 2024-09-19T06:08:32.763 INFO:teuthology.orchestra.run.smithi155.stdout:rbd-mirror-19.1.1-361.g2f599a46.el9.x86_64.rpm 9.2 MB/s | 3.1 MB 00:00 2024-09-19T06:08:32.764 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:08:32.764 INFO:teuthology.orchestra.run.smithi155.stdout:Total 9.1 MB/s | 3.1 MB 00:00 2024-09-19T06:08:32.765 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-09-19T06:08:32.773 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-09-19T06:08:32.774 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-09-19T06:08:32.855 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-09-19T06:08:32.856 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-09-19T06:08:33.222 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:59 ago on Thu 19 Sep 2024 06:06:34 AM UTC. 2024-09-19T06:08:33.293 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-09-19T06:08:33.316 INFO:teuthology.orchestra.run.smithi053.stdout:Package python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 is already installed. 2024-09-19T06:08:33.361 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-19T06:08:33.362 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-19T06:08:33.362 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-19T06:08:33.433 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install rbd-fuse 2024-09-19T06:08:33.473 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 1/1 2024-09-19T06:08:33.499 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 1/1 2024-09-19T06:08:33.500 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:08:33.500 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T06:08:33.500 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T06:08:33.500 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T06:08:33.500 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:34.036 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:02:00 ago on Thu 19 Sep 2024 06:06:34 AM UTC. 2024-09-19T06:08:34.174 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-19T06:08:34.176 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:08:34.176 INFO:teuthology.orchestra.run.smithi053.stdout: Package Architecture Version Repository Size 2024-09-19T06:08:34.176 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:08:34.176 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-19T06:08:34.176 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T06:08:34.177 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:34.177 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-19T06:08:34.177 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:08:34.177 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-09-19T06:08:34.177 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:34.177 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 85 k 2024-09-19T06:08:34.178 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 231 k 2024-09-19T06:08:34.178 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-19T06:08:34.376 INFO:teuthology.orchestra.run.smithi053.stdout:rbd-fuse-19.1.1-361.g2f599a46.el9.x86_64.rpm 426 kB/s | 85 kB 00:00 2024-09-19T06:08:34.377 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:08:34.377 INFO:teuthology.orchestra.run.smithi053.stdout:Total 423 kB/s | 85 kB 00:00 2024-09-19T06:08:34.377 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-19T06:08:34.385 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-19T06:08:34.385 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-19T06:08:34.453 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-19T06:08:34.454 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-19T06:08:34.481 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 1/1 2024-09-19T06:08:34.481 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:34.481 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-09-19T06:08:34.481 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:08:34.481 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:34.481 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-09-19T06:08:34.601 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-19T06:08:34.719 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 1/1 2024-09-19T06:08:34.735 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install rbd-nbd 2024-09-19T06:08:35.094 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 1/1 2024-09-19T06:08:35.327 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:50 ago on Thu 19 Sep 2024 06:06:45 AM UTC. 2024-09-19T06:08:35.360 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 1/1 2024-09-19T06:08:35.360 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:35.360 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-19T06:08:35.361 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:08:35.361 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:35.361 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-19T06:08:35.460 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-09-19T06:08:35.461 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:08:35.461 INFO:teuthology.orchestra.run.smithi155.stdout: Package Architecture Version Repository Size 2024-09-19T06:08:35.461 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:08:35.461 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-09-19T06:08:35.461 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T06:08:35.461 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:35.462 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-09-19T06:08:35.462 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:08:35.462 INFO:teuthology.orchestra.run.smithi155.stdout:Install 1 Package 2024-09-19T06:08:35.462 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:35.462 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 171 k 2024-09-19T06:08:35.462 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 494 k 2024-09-19T06:08:35.462 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-09-19T06:08:35.630 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install rbd-mirror 2024-09-19T06:08:35.668 INFO:teuthology.orchestra.run.smithi155.stdout:rbd-nbd-19.1.1-361.g2f599a46.el9.x86_64.rpm 828 kB/s | 171 kB 00:00 2024-09-19T06:08:35.669 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:08:35.669 INFO:teuthology.orchestra.run.smithi155.stdout:Total 822 kB/s | 171 kB 00:00 2024-09-19T06:08:35.670 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-09-19T06:08:35.677 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-09-19T06:08:35.678 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-09-19T06:08:35.741 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-09-19T06:08:35.741 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-09-19T06:08:35.898 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-09-19T06:08:36.015 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 1/1 2024-09-19T06:08:36.245 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:02:02 ago on Thu 19 Sep 2024 06:06:34 AM UTC. 2024-09-19T06:08:36.384 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-19T06:08:36.385 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:08:36.385 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repo Size 2024-09-19T06:08:36.385 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:08:36.385 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-19T06:08:36.385 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T06:08:36.386 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:36.386 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-19T06:08:36.386 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:08:36.386 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-09-19T06:08:36.386 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:36.386 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 3.1 M 2024-09-19T06:08:36.386 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 13 M 2024-09-19T06:08:36.386 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-19T06:08:36.466 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 1/1 2024-09-19T06:08:36.748 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 1/1 2024-09-19T06:08:36.748 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:36.749 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-09-19T06:08:36.749 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:08:36.749 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:08:36.749 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-09-19T06:08:37.001 DEBUG:teuthology.parallel:result is None 2024-09-19T06:08:37.263 INFO:teuthology.orchestra.run.smithi053.stdout:rbd-mirror-19.1.1-361.g2f599a46.el9.x86_64.rpm 3.5 MB/s | 3.1 MB 00:00 2024-09-19T06:08:37.264 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:08:37.264 INFO:teuthology.orchestra.run.smithi053.stdout:Total 3.5 MB/s | 3.1 MB 00:00 2024-09-19T06:08:37.264 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-19T06:08:37.275 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-19T06:08:37.276 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-19T06:08:37.364 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-19T06:08:37.364 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-19T06:08:37.823 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-19T06:08:37.889 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 1/1 2024-09-19T06:08:37.915 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 1/1 2024-09-19T06:08:37.916 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:08:37.916 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T06:08:37.916 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T06:08:37.916 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T06:08:37.916 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:39.046 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 1/1 2024-09-19T06:08:39.046 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:39.046 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-19T06:08:39.046 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:08:39.046 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:39.046 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-19T06:08:39.293 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install rbd-nbd 2024-09-19T06:08:39.898 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:02:05 ago on Thu 19 Sep 2024 06:06:34 AM UTC. 2024-09-19T06:08:40.036 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-19T06:08:40.037 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:08:40.037 INFO:teuthology.orchestra.run.smithi053.stdout: Package Architecture Version Repository Size 2024-09-19T06:08:40.037 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:08:40.037 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-19T06:08:40.037 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T06:08:40.037 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:40.037 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-19T06:08:40.038 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:08:40.038 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-09-19T06:08:40.038 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:40.038 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 171 k 2024-09-19T06:08:40.038 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 494 k 2024-09-19T06:08:40.038 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-19T06:08:40.259 INFO:teuthology.orchestra.run.smithi053.stdout:rbd-nbd-19.1.1-361.g2f599a46.el9.x86_64.rpm 774 kB/s | 171 kB 00:00 2024-09-19T06:08:40.259 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:08:40.259 INFO:teuthology.orchestra.run.smithi053.stdout:Total 769 kB/s | 171 kB 00:00 2024-09-19T06:08:40.260 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-19T06:08:40.268 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-19T06:08:40.268 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-19T06:08:40.336 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-19T06:08:40.337 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-19T06:08:40.493 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-19T06:08:40.593 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 1/1 2024-09-19T06:08:41.017 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 1/1 2024-09-19T06:08:41.277 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 1/1 2024-09-19T06:08:41.277 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:41.277 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-19T06:08:41.277 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:08:41.277 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:08:41.277 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-19T06:08:41.546 DEBUG:teuthology.parallel:result is None 2024-09-19T06:08:41.547 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:08:41.697 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T06:08:41.719 INFO:teuthology.orchestra.run.smithi053.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T06:08:41.720 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T06:08:41.720 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T06:08:41.721 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:08:41.862 DEBUG:teuthology.orchestra.run.smithi155:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T06:08:41.888 INFO:teuthology.orchestra.run.smithi155.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T06:08:41.888 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T06:08:41.888 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T06:08:41.890 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-19T06:08:41.890 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-19T06:08:41.891 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T06:08:41.918 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-19T06:08:41.918 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T06:08:41.959 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-19T06:08:41.959 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-19T06:08:41.959 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T06:08:41.984 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T06:08:42.050 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-19T06:08:42.080 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T06:08:42.110 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T06:08:42.181 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-19T06:08:42.181 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-19T06:08:42.181 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T06:08:42.207 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T06:08:42.273 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-19T06:08:42.273 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T06:08:42.303 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T06:08:42.372 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-19T06:08:42.372 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-19T06:08:42.372 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T06:08:42.397 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T06:08:42.465 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-19T06:08:42.465 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T06:08:42.494 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T06:08:42.561 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-19T06:08:42.655 INFO:tasks.cephadm:Config: {'roleless': True, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd op complaint time': 180, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'} 2024-09-19T06:08:42.655 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:08:42.656 INFO:tasks.cephadm:Cluster fsid is 9f2876d0-764d-11ef-bceb-c7b262605968 2024-09-19T06:08:42.656 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-19T06:08:42.656 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-09-19T06:08:42.656 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi053': '172.21.15.53', 'mon.smithi155': '172.21.15.155'} 2024-09-19T06:08:42.656 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-19T06:08:42.657 DEBUG:teuthology.orchestra.run.smithi053:> sudo hostname $(hostname -s) 2024-09-19T06:08:42.683 DEBUG:teuthology.orchestra.run.smithi155:> sudo hostname $(hostname -s) 2024-09-19T06:08:42.712 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-19T06:08:42.713 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:08:42.852 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56106/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-361-g2f599a46', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-361.g2f599a46'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-18 17:52:24.287651', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-19T06:08:42.992 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref squid, sha1 2f599a46ad10e37c929e5c43da2f6907ddcd1961 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:08:42.993 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T06:08:42.994 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T06:08:42.994 DEBUG:teuthology.orchestra.run.smithi053:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T06:08:43.379 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 06:08 /home/ubuntu/cephtest/cephadm 2024-09-19T06:08:43.380 DEBUG:teuthology.orchestra.run.smithi155:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T06:08:43.758 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 06:08 /home/ubuntu/cephtest/cephadm 2024-09-19T06:08:43.758 DEBUG:teuthology.orchestra.run.smithi053:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T06:08:43.779 DEBUG:teuthology.orchestra.run.smithi155:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T06:08:43.808 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 on all hosts... 2024-09-19T06:08:43.808 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T06:08:43.821 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T06:08:45.475 INFO:teuthology.orchestra.run.smithi155.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T06:08:45.504 INFO:teuthology.orchestra.run.smithi053.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T06:09:13.445 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-19T06:09:13.446 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T06:09:13.446 INFO:teuthology.orchestra.run.smithi053.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T06:09:13.446 INFO:teuthology.orchestra.run.smithi053.stdout: "repo_digests": [ 2024-09-19T06:09:13.446 INFO:teuthology.orchestra.run.smithi053.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T06:09:13.446 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-19T06:09:13.446 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-19T06:09:17.856 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-19T06:09:17.857 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T06:09:17.857 INFO:teuthology.orchestra.run.smithi155.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T06:09:17.857 INFO:teuthology.orchestra.run.smithi155.stdout: "repo_digests": [ 2024-09-19T06:09:17.857 INFO:teuthology.orchestra.run.smithi155.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T06:09:17.857 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-19T06:09:17.857 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-19T06:09:17.882 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /etc/ceph 2024-09-19T06:09:17.917 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /etc/ceph 2024-09-19T06:09:17.944 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 777 /etc/ceph 2024-09-19T06:09:17.986 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 777 /etc/ceph 2024-09-19T06:09:18.011 INFO:tasks.cephadm:Writing seed config... 2024-09-19T06:09:18.012 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-19T06:09:18.012 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-19T06:09:18.012 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-19T06:09:18.012 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-19T06:09:18.012 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-19T06:09:18.012 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-19T06:09:18.012 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-19T06:09:18.012 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-19T06:09:18.012 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-19T06:09:18.013 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-19T06:09:18.013 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-19T06:09:18.013 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-19T06:09:18.013 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-19T06:09:18.013 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-19T06:09:18.013 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-19T06:09:18.013 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-19T06:09:18.013 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-19T06:09:18.013 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-19T06:09:18.013 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-19T06:09:18.013 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-19T06:09:18.013 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-19T06:09:18.013 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-19T06:09:18.013 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-19T06:09:18.014 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-19T06:09:18.014 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-19T06:09:18.014 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-19T06:09:18.014 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-19T06:09:18.014 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-19T06:09:18.014 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-19T06:09:18.015 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-19T06:09:18.046 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 9f2876d0-764d-11ef-bceb-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-19T06:09:18.046 DEBUG:teuthology.orchestra.run.smithi053:mon.smithi053> sudo journalctl -f -n 0 -u ceph-9f2876d0-764d-11ef-bceb-c7b262605968@mon.smithi053.service 2024-09-19T06:09:18.088 INFO:tasks.cephadm:Bootstrapping... 2024-09-19T06:09:18.088 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 -v bootstrap --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.53 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-19T06:09:18.305 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:09:18.306 INFO:teuthology.orchestra.run.smithi053.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961', '-v', 'bootstrap', '--fsid', '9f2876d0-764d-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.53', '--skip-admin-label'] 2024-09-19T06:09:18.306 INFO:teuthology.orchestra.run.smithi053.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-19T06:09:18.307 INFO:teuthology.orchestra.run.smithi053.stdout:Verifying podman|docker is present... 2024-09-19T06:09:18.331 INFO:teuthology.orchestra.run.smithi053.stdout:/bin/podman: stdout 5.2.2 2024-09-19T06:09:18.331 INFO:teuthology.orchestra.run.smithi053.stdout:Verifying lvm2 is present... 2024-09-19T06:09:18.331 INFO:teuthology.orchestra.run.smithi053.stdout:Verifying time synchronization is in place... 2024-09-19T06:09:18.340 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T06:09:18.340 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T06:09:18.346 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T06:09:18.346 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout inactive 2024-09-19T06:09:18.353 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout enabled 2024-09-19T06:09:18.360 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout active 2024-09-19T06:09:18.360 INFO:teuthology.orchestra.run.smithi053.stdout:Unit chronyd.service is enabled and running 2024-09-19T06:09:18.360 INFO:teuthology.orchestra.run.smithi053.stdout:Repeating the final host check... 2024-09-19T06:09:18.384 INFO:teuthology.orchestra.run.smithi053.stdout:/bin/podman: stdout 5.2.2 2024-09-19T06:09:18.384 INFO:teuthology.orchestra.run.smithi053.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-19T06:09:18.384 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl is present 2024-09-19T06:09:18.384 INFO:teuthology.orchestra.run.smithi053.stdout:lvcreate is present 2024-09-19T06:09:18.391 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T06:09:18.391 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T06:09:18.397 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T06:09:18.397 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout inactive 2024-09-19T06:09:18.404 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout enabled 2024-09-19T06:09:18.411 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout active 2024-09-19T06:09:18.411 INFO:teuthology.orchestra.run.smithi053.stdout:Unit chronyd.service is enabled and running 2024-09-19T06:09:18.411 INFO:teuthology.orchestra.run.smithi053.stdout:Host looks OK 2024-09-19T06:09:18.411 INFO:teuthology.orchestra.run.smithi053.stdout:Cluster fsid: 9f2876d0-764d-11ef-bceb-c7b262605968 2024-09-19T06:09:18.411 INFO:teuthology.orchestra.run.smithi053.stdout:Acquiring lock 139753522984800 on /run/cephadm/9f2876d0-764d-11ef-bceb-c7b262605968.lock 2024-09-19T06:09:18.412 INFO:teuthology.orchestra.run.smithi053.stdout:Lock 139753522984800 acquired on /run/cephadm/9f2876d0-764d-11ef-bceb-c7b262605968.lock 2024-09-19T06:09:18.412 INFO:teuthology.orchestra.run.smithi053.stdout:Verifying IP 172.21.15.53 port 3300 ... 2024-09-19T06:09:18.412 INFO:teuthology.orchestra.run.smithi053.stdout:Verifying IP 172.21.15.53 port 6789 ... 2024-09-19T06:09:18.412 INFO:teuthology.orchestra.run.smithi053.stdout:Base mon IP(s) is [172.21.15.53:3300, 172.21.15.53:6789], mon addrv is [v2:172.21.15.53:3300,v1:172.21.15.53:6789] 2024-09-19T06:09:18.415 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.53 metric 100 2024-09-19T06:09:18.415 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.53 metric 100 2024-09-19T06:09:18.418 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-19T06:09:18.418 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-19T06:09:18.418 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1797sec hoplimit 64 pref medium 2024-09-19T06:09:18.420 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-19T06:09:18.420 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-19T06:09:18.420 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T06:09:18.421 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-19T06:09:18.421 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:3718/64 scope link 2024-09-19T06:09:18.421 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T06:09:18.421 INFO:teuthology.orchestra.run.smithi053.stdout:Mon IP `172.21.15.53` is in CIDR network `172.21.0.0/20` 2024-09-19T06:09:18.421 INFO:teuthology.orchestra.run.smithi053.stdout:Mon IP `172.21.15.53` is in CIDR network `172.21.0.0/20` 2024-09-19T06:09:18.422 INFO:teuthology.orchestra.run.smithi053.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-19T06:09:18.422 INFO:teuthology.orchestra.run.smithi053.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-19T06:09:18.422 INFO:teuthology.orchestra.run.smithi053.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T06:09:18.898 INFO:teuthology.orchestra.run.smithi053.stdout:/bin/podman: stdout d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T06:09:18.898 INFO:teuthology.orchestra.run.smithi053.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T06:09:18.898 INFO:teuthology.orchestra.run.smithi053.stdout:/bin/podman: stderr Getting image source signatures 2024-09-19T06:09:18.898 INFO:teuthology.orchestra.run.smithi053.stdout:/bin/podman: stderr Copying blob sha256:c4221751d16d26d2a4b7031ebd71ecffffff13c8cf7401cf95ce80dab3b6d67a 2024-09-19T06:09:18.898 INFO:teuthology.orchestra.run.smithi053.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-19T06:09:18.898 INFO:teuthology.orchestra.run.smithi053.stdout:/bin/podman: stderr Copying config sha256:d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T06:09:18.898 INFO:teuthology.orchestra.run.smithi053.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-19T06:09:19.819 INFO:teuthology.orchestra.run.smithi053.stdout:ceph: stdout ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T06:09:19.819 INFO:teuthology.orchestra.run.smithi053.stdout:Ceph version: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T06:09:19.820 INFO:teuthology.orchestra.run.smithi053.stdout:Extracting ceph user uid/gid from container image... 2024-09-19T06:09:20.818 INFO:teuthology.orchestra.run.smithi053.stdout:stat: stdout 167 167 2024-09-19T06:09:20.818 INFO:teuthology.orchestra.run.smithi053.stdout:Creating initial keys... 2024-09-19T06:09:21.665 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph-authtool: stdout AQARwOtmK6QFCBAA3tUC0wetCuEuLV7V82wTDA== 2024-09-19T06:09:22.563 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph-authtool: stdout AQARwOtmj9IEOhAAqGWnl60ugL5D7qbDY8mnUA== 2024-09-19T06:09:23.494 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph-authtool: stdout AQASwOtmal5oNBAAwJamtEjbHTJXyY/juU6a4A== 2024-09-19T06:09:23.495 INFO:teuthology.orchestra.run.smithi053.stdout:Creating initial monmap... 2024-09-19T06:09:24.384 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T06:09:24.384 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-19T06:09:24.384 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 9f2876d0-764d-11ef-bceb-c7b262605968 2024-09-19T06:09:24.384 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T06:09:24.384 INFO:teuthology.orchestra.run.smithi053.stdout:monmaptool for smithi053 [v2:172.21.15.53:3300,v1:172.21.15.53:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T06:09:24.384 INFO:teuthology.orchestra.run.smithi053.stdout:setting min_mon_release = quincy 2024-09-19T06:09:24.384 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/monmaptool: set fsid to 9f2876d0-764d-11ef-bceb-c7b262605968 2024-09-19T06:09:24.384 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T06:09:24.385 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:09:24.385 INFO:teuthology.orchestra.run.smithi053.stdout:Creating mon... 2024-09-19T06:09:25.306 INFO:teuthology.orchestra.run.smithi053.stdout:create mon.smithi053 on 2024-09-19T06:09:25.519 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-19T06:09:25.685 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-19T06:09:25.858 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-9f2876d0-764d-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-9f2876d0-764d-11ef-bceb-c7b262605968.target. 2024-09-19T06:09:25.859 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-9f2876d0-764d-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-9f2876d0-764d-11ef-bceb-c7b262605968.target. 2024-09-19T06:09:26.100 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9f2876d0-764d-11ef-bceb-c7b262605968@mon.smithi053 2024-09-19T06:09:26.100 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9f2876d0-764d-11ef-bceb-c7b262605968@mon.smithi053.service: Unit ceph-9f2876d0-764d-11ef-bceb-c7b262605968@mon.smithi053.service not loaded. 2024-09-19T06:09:26.274 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9f2876d0-764d-11ef-bceb-c7b262605968.target.wants/ceph-9f2876d0-764d-11ef-bceb-c7b262605968@mon.smithi053.service → /etc/systemd/system/ceph-9f2876d0-764d-11ef-bceb-c7b262605968@.service. 2024-09-19T06:09:26.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:26 smithi053 podman[27586]: 2024-09-19 06:09:26.422369955 +0000 UTC m=+0.017071747 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:09:26.851 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T06:09:26.851 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout disabled 2024-09-19T06:09:26.860 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T06:09:26.860 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout inactive 2024-09-19T06:09:26.860 INFO:teuthology.orchestra.run.smithi053.stdout:firewalld.service is not enabled 2024-09-19T06:09:26.860 INFO:teuthology.orchestra.run.smithi053.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T06:09:26.860 INFO:teuthology.orchestra.run.smithi053.stdout:Waiting for mon to start... 2024-09-19T06:09:26.860 INFO:teuthology.orchestra.run.smithi053.stdout:Waiting for mon... 2024-09-19T06:09:26.901 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:26 smithi053 podman[27586]: 2024-09-19 06:09:26.756874768 +0000 UTC m=+0.351576568 container init d5b2f70151d8ddc7cdf78b1e2a71db6161a1056665776c534832f8da218b2454 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-mon-smithi053, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-2f599a4, io.buildah.version=1.37.2) 2024-09-19T06:09:26.902 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:26 smithi053 podman[27586]: 2024-09-19 06:09:26.760432743 +0000 UTC m=+0.355134534 container start d5b2f70151d8ddc7cdf78b1e2a71db6161a1056665776c534832f8da218b2454 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-mon-smithi053, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-19T06:09:26.902 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:26 smithi053 ceph-mon[27605]: mkfs 9f2876d0-764d-11ef-bceb-c7b262605968 2024-09-19T06:09:26.902 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:26 smithi053 ceph-mon[27605]: mon.smithi053 is new leader, mons smithi053 in quorum (ranks 0) 2024-09-19T06:09:26.902 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:26 smithi053 bash[27586]: d5b2f70151d8ddc7cdf78b1e2a71db6161a1056665776c534832f8da218b2454 2024-09-19T06:09:26.902 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:26 smithi053 systemd[1]: Started Ceph mon.smithi053 for 9f2876d0-764d-11ef-bceb-c7b262605968. 2024-09-19T06:09:27.975 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout cluster: 2024-09-19T06:09:27.976 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout id: 9f2876d0-764d-11ef-bceb-c7b262605968 2024-09-19T06:09:27.976 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-19T06:09:27.976 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2024-09-19T06:09:27.976 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout services: 2024-09-19T06:09:27.976 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi053 (age 0.581983s) 2024-09-19T06:09:27.976 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-19T06:09:27.976 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-19T06:09:27.976 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2024-09-19T06:09:27.976 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout data: 2024-09-19T06:09:27.976 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-19T06:09:27.976 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-19T06:09:27.976 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-19T06:09:27.977 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout pgs: 2024-09-19T06:09:27.977 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2024-09-19T06:09:27.977 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.306+0000 7f14066fb640 1 Processor -- start 2024-09-19T06:09:27.977 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.307+0000 7f14066fb640 1 -- start start 2024-09-19T06:09:27.977 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.307+0000 7f14066fb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14001059d0 0x7f1400105dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:27.977 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.307+0000 7f14066fb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f14001063a0 con 0x7f14001059d0 2024-09-19T06:09:27.977 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.307+0000 7f13fffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14001059d0 0x7f1400105dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:27.977 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.307+0000 7f13fffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14001059d0 0x7f1400105dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36644/0 (socket says 172.21.15.53:36644) 2024-09-19T06:09:27.977 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.307+0000 7f13fffff640 1 -- 172.21.15.53:0/516518891 learned_addr learned my addr 172.21.15.53:0/516518891 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:09:27.977 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.308+0000 7f13fffff640 1 -- 172.21.15.53:0/516518891 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1400106bd0 con 0x7f14001059d0 2024-09-19T06:09:27.977 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.310+0000 7f13fffff640 1 --2- 172.21.15.53:0/516518891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14001059d0 0x7f1400105dd0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f13f0009820 tx=0x7f13f002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeff7f2d53f2d2f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:27.977 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.310+0000 7f13feffd640 1 -- 172.21.15.53:0/516518891 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f13f0037070 con 0x7f14001059d0 2024-09-19T06:09:27.977 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.310+0000 7f13feffd640 1 -- 172.21.15.53:0/516518891 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13f002ad40 con 0x7f14001059d0 2024-09-19T06:09:27.978 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.310+0000 7f13feffd640 1 -- 172.21.15.53:0/516518891 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f13f00325b0 con 0x7f14001059d0 2024-09-19T06:09:27.978 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.311+0000 7f14066fb640 1 -- 172.21.15.53:0/516518891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14001059d0 msgr2=0x7f1400105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:27.978 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.311+0000 7f14066fb640 1 --2- 172.21.15.53:0/516518891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14001059d0 0x7f1400105dd0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f13f0009820 tx=0x7f13f002a270 comp rx=0 tx=0).stop 2024-09-19T06:09:27.978 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.311+0000 7f14066fb640 1 -- 172.21.15.53:0/516518891 shutdown_connections 2024-09-19T06:09:27.978 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.311+0000 7f14066fb640 1 --2- 172.21.15.53:0/516518891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14001059d0 0x7f1400105dd0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.978 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.311+0000 7f14066fb640 1 -- 172.21.15.53:0/516518891 >> 172.21.15.53:0/516518891 conn(0x7f14001011a0 msgr2=0x7f14001035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:27.978 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.311+0000 7f14066fb640 1 -- 172.21.15.53:0/516518891 shutdown_connections 2024-09-19T06:09:27.978 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.311+0000 7f14066fb640 1 -- 172.21.15.53:0/516518891 wait complete. 2024-09-19T06:09:27.978 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.311+0000 7f14066fb640 1 Processor -- start 2024-09-19T06:09:27.978 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.311+0000 7f14066fb640 1 -- start start 2024-09-19T06:09:27.978 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.312+0000 7f14066fb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14001059d0 0x7f140019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:27.978 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.312+0000 7f14066fb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1400107170 con 0x7f14001059d0 2024-09-19T06:09:27.979 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.312+0000 7f13fffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14001059d0 0x7f140019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:27.979 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.312+0000 7f13fffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14001059d0 0x7f140019a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36654/0 (socket says 172.21.15.53:36654) 2024-09-19T06:09:27.979 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.312+0000 7f13fffff640 1 -- 172.21.15.53:0/1020917470 learned_addr learned my addr 172.21.15.53:0/1020917470 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:09:27.979 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.312+0000 7f13fffff640 1 -- 172.21.15.53:0/1020917470 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f140019a7d0 con 0x7f14001059d0 2024-09-19T06:09:27.979 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.312+0000 7f13fffff640 1 --2- 172.21.15.53:0/1020917470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14001059d0 0x7f140019a290 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f13f00090b0 tx=0x7f13f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:27.979 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.312+0000 7f13fd7fa640 1 -- 172.21.15.53:0/1020917470 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f13f003a070 con 0x7f14001059d0 2024-09-19T06:09:27.979 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.312+0000 7f13fd7fa640 1 -- 172.21.15.53:0/1020917470 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13f00305d0 con 0x7f14001059d0 2024-09-19T06:09:27.979 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.312+0000 7f13fd7fa640 1 -- 172.21.15.53:0/1020917470 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f13f0030910 con 0x7f14001059d0 2024-09-19T06:09:27.979 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.312+0000 7f14066fb640 1 -- 172.21.15.53:0/1020917470 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f140019aa60 con 0x7f14001059d0 2024-09-19T06:09:27.979 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.312+0000 7f14066fb640 1 -- 172.21.15.53:0/1020917470 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f140019d750 con 0x7f14001059d0 2024-09-19T06:09:27.979 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.313+0000 7f13fd7fa640 1 -- 172.21.15.53:0/1020917470 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f13f0030ab0 con 0x7f14001059d0 2024-09-19T06:09:27.979 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.313+0000 7f13fd7fa640 1 -- 172.21.15.53:0/1020917470 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f13f0049740 con 0x7f14001059d0 2024-09-19T06:09:27.980 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.313+0000 7f14066fb640 1 -- 172.21.15.53:0/1020917470 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14001077c0 con 0x7f14001059d0 2024-09-19T06:09:27.980 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.316+0000 7f13fd7fa640 1 -- 172.21.15.53:0/1020917470 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f13f0040cf0 con 0x7f14001059d0 2024-09-19T06:09:27.980 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.377+0000 7f14066fb640 1 -- 172.21.15.53:0/1020917470 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f140002d910 con 0x7f14001059d0 2024-09-19T06:09:27.980 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.378+0000 7f13fd7fa640 1 -- 172.21.15.53:0/1020917470 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+325 (secure 0 0 0) 0x7f13f0040440 con 0x7f14001059d0 2024-09-19T06:09:27.980 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.379+0000 7f14066fb640 1 -- 172.21.15.53:0/1020917470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14001059d0 msgr2=0x7f140019a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:27.980 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.379+0000 7f14066fb640 1 --2- 172.21.15.53:0/1020917470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14001059d0 0x7f140019a290 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f13f00090b0 tx=0x7f13f00046c0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.980 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.379+0000 7f14066fb640 1 -- 172.21.15.53:0/1020917470 shutdown_connections 2024-09-19T06:09:27.980 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.379+0000 7f14066fb640 1 --2- 172.21.15.53:0/1020917470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14001059d0 0x7f140019a290 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.980 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.379+0000 7f14066fb640 1 -- 172.21.15.53:0/1020917470 >> 172.21.15.53:0/1020917470 conn(0x7f14001011a0 msgr2=0x7f1400190f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:27.980 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.379+0000 7f14066fb640 1 -- 172.21.15.53:0/1020917470 shutdown_connections 2024-09-19T06:09:27.980 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:27.379+0000 7f14066fb640 1 -- 172.21.15.53:0/1020917470 wait complete. 2024-09-19T06:09:27.980 INFO:teuthology.orchestra.run.smithi053.stdout:mon is available 2024-09-19T06:09:27.980 INFO:teuthology.orchestra.run.smithi053.stdout:Assimilating anything we can from ceph.conf... 2024-09-19T06:09:28.082 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:27 smithi053 ceph-mon[27605]: mon.smithi053 is new leader, mons smithi053 in quorum (ranks 0) 2024-09-19T06:09:28.082 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:27 smithi053 ceph-mon[27605]: monmap epoch 1 2024-09-19T06:09:28.082 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:27 smithi053 ceph-mon[27605]: fsid 9f2876d0-764d-11ef-bceb-c7b262605968 2024-09-19T06:09:28.083 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:27 smithi053 ceph-mon[27605]: last_changed 2024-09-19T06:09:23.791881+0000 2024-09-19T06:09:28.083 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:27 smithi053 ceph-mon[27605]: created 2024-09-19T06:09:23.791881+0000 2024-09-19T06:09:28.083 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:27 smithi053 ceph-mon[27605]: min_mon_release 19 (squid) 2024-09-19T06:09:28.083 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:27 smithi053 ceph-mon[27605]: election_strategy: 1 2024-09-19T06:09:28.083 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:27 smithi053 ceph-mon[27605]: 0: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.smithi053 2024-09-19T06:09:28.083 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:27 smithi053 ceph-mon[27605]: fsmap 2024-09-19T06:09:28.083 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:27 smithi053 ceph-mon[27605]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T06:09:28.083 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:27 smithi053 ceph-mon[27605]: mgrmap e1: no daemons active 2024-09-19T06:09:28.083 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:27 smithi053 ceph-mon[27605]: from='client.? 172.21.15.53:0/1020917470' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-19T06:09:29.013 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:28 smithi053 ceph-mon[27605]: from='client.? 172.21.15.53:0/1619998599' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T06:09:29.013 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:28 smithi053 ceph-mon[27605]: from='client.? 172.21.15.53:0/1619998599' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-19T06:09:29.021 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2024-09-19T06:09:29.021 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T06:09:29.021 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout fsid = 9f2876d0-764d-11ef-bceb-c7b262605968 2024-09-19T06:09:29.021 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.53:3300,v1:172.21.15.53:6789] 2024-09-19T06:09:29.021 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T06:09:29.021 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T06:09:29.021 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T06:09:29.021 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T06:09:29.021 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2024-09-19T06:09:29.022 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T06:09:29.022 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T06:09:29.022 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2024-09-19T06:09:29.022 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T06:09:29.022 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T06:09:29.022 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T06:09:29.022 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.374+0000 7feb28e22640 1 Processor -- start 2024-09-19T06:09:29.022 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.374+0000 7feb28e22640 1 -- start start 2024-09-19T06:09:29.022 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.375+0000 7feb28e22640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb24107580 0x7feb24109970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:29.022 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.375+0000 7feb28e22640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7feb2407a6f0 con 0x7feb24107580 2024-09-19T06:09:29.022 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.375+0000 7feb2259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb24107580 0x7feb24109970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:29.022 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.375+0000 7feb2259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb24107580 0x7feb24109970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36666/0 (socket says 172.21.15.53:36666) 2024-09-19T06:09:29.022 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.375+0000 7feb2259c640 1 -- 172.21.15.53:0/2725923326 learned_addr learned my addr 172.21.15.53:0/2725923326 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:09:29.022 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.375+0000 7feb2259c640 1 -- 172.21.15.53:0/2725923326 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb24109eb0 con 0x7feb24107580 2024-09-19T06:09:29.022 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.376+0000 7feb2259c640 1 --2- 172.21.15.53:0/2725923326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb24107580 0x7feb24109970 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7feb14009820 tx=0x7feb1402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1720069f12a5a055 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:29.023 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.376+0000 7feb2159a640 1 -- 172.21.15.53:0/2725923326 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7feb14037070 con 0x7feb24107580 2024-09-19T06:09:29.023 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.376+0000 7feb2159a640 1 -- 172.21.15.53:0/2725923326 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb1402ad40 con 0x7feb24107580 2024-09-19T06:09:29.023 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.376+0000 7feb2159a640 1 -- 172.21.15.53:0/2725923326 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7feb140325b0 con 0x7feb24107580 2024-09-19T06:09:29.023 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.377+0000 7feb28e22640 1 -- 172.21.15.53:0/2725923326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb24107580 msgr2=0x7feb24109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:29.023 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.377+0000 7feb28e22640 1 --2- 172.21.15.53:0/2725923326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb24107580 0x7feb24109970 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7feb14009820 tx=0x7feb1402a270 comp rx=0 tx=0).stop 2024-09-19T06:09:29.023 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.377+0000 7feb28e22640 1 -- 172.21.15.53:0/2725923326 shutdown_connections 2024-09-19T06:09:29.023 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.377+0000 7feb28e22640 1 --2- 172.21.15.53:0/2725923326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb24107580 0x7feb24109970 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:29.023 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.377+0000 7feb28e22640 1 -- 172.21.15.53:0/2725923326 >> 172.21.15.53:0/2725923326 conn(0x7feb241011a0 msgr2=0x7feb241035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:29.023 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.377+0000 7feb28e22640 1 -- 172.21.15.53:0/2725923326 shutdown_connections 2024-09-19T06:09:29.023 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.377+0000 7feb28e22640 1 -- 172.21.15.53:0/2725923326 wait complete. 2024-09-19T06:09:29.023 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.378+0000 7feb28e22640 1 Processor -- start 2024-09-19T06:09:29.023 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.378+0000 7feb28e22640 1 -- start start 2024-09-19T06:09:29.023 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.378+0000 7feb28e22640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb24107580 0x7feb2419a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:29.023 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.378+0000 7feb28e22640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7feb2410a390 con 0x7feb24107580 2024-09-19T06:09:29.023 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.378+0000 7feb2259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb24107580 0x7feb2419a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:29.024 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.378+0000 7feb2259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb24107580 0x7feb2419a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36674/0 (socket says 172.21.15.53:36674) 2024-09-19T06:09:29.024 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.378+0000 7feb2259c640 1 -- 172.21.15.53:0/1619998599 learned_addr learned my addr 172.21.15.53:0/1619998599 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:09:29.024 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.379+0000 7feb2259c640 1 -- 172.21.15.53:0/1619998599 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb2419a7e0 con 0x7feb24107580 2024-09-19T06:09:29.024 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.379+0000 7feb2259c640 1 --2- 172.21.15.53:0/1619998599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb24107580 0x7feb2419a2a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7feb140090b0 tx=0x7feb140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:29.024 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.379+0000 7feb1b7fe640 1 -- 172.21.15.53:0/1619998599 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7feb1403a070 con 0x7feb24107580 2024-09-19T06:09:29.024 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.379+0000 7feb1b7fe640 1 -- 172.21.15.53:0/1619998599 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb140305d0 con 0x7feb24107580 2024-09-19T06:09:29.024 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.379+0000 7feb28e22640 1 -- 172.21.15.53:0/1619998599 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb2419aa70 con 0x7feb24107580 2024-09-19T06:09:29.024 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.379+0000 7feb1b7fe640 1 -- 172.21.15.53:0/1619998599 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7feb14030910 con 0x7feb24107580 2024-09-19T06:09:29.024 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.379+0000 7feb28e22640 1 -- 172.21.15.53:0/1619998599 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb2419d760 con 0x7feb24107580 2024-09-19T06:09:29.024 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.379+0000 7feb1b7fe640 1 -- 172.21.15.53:0/1619998599 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7feb14030ab0 con 0x7feb24107580 2024-09-19T06:09:29.024 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.379+0000 7feb1b7fe640 1 -- 172.21.15.53:0/1619998599 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7feb14049790 con 0x7feb24107580 2024-09-19T06:09:29.024 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.380+0000 7feb28e22640 1 -- 172.21.15.53:0/1619998599 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb241079e0 con 0x7feb24107580 2024-09-19T06:09:29.024 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.382+0000 7feb1b7fe640 1 -- 172.21.15.53:0/1619998599 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7feb14040440 con 0x7feb24107580 2024-09-19T06:09:29.024 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.443+0000 7feb28e22640 1 -- 172.21.15.53:0/1619998599 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7feb2402d910 con 0x7feb24107580 2024-09-19T06:09:29.024 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.446+0000 7feb1b7fe640 1 -- 172.21.15.53:0/1619998599 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+338 (secure 0 0 0) 0x7feb14049990 con 0x7feb24107580 2024-09-19T06:09:29.025 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.446+0000 7feb1b7fe640 1 -- 172.21.15.53:0/1619998599 <== mon.0 v2:172.21.15.53:3300/0 8 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7feb1402ad40 con 0x7feb24107580 2024-09-19T06:09:29.025 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.447+0000 7feb28e22640 1 -- 172.21.15.53:0/1619998599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb24107580 msgr2=0x7feb2419a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:29.025 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.447+0000 7feb28e22640 1 --2- 172.21.15.53:0/1619998599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb24107580 0x7feb2419a2a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7feb140090b0 tx=0x7feb140046c0 comp rx=0 tx=0).stop 2024-09-19T06:09:29.025 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.447+0000 7feb28e22640 1 -- 172.21.15.53:0/1619998599 shutdown_connections 2024-09-19T06:09:29.025 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.447+0000 7feb28e22640 1 --2- 172.21.15.53:0/1619998599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb24107580 0x7feb2419a2a0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:29.025 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.447+0000 7feb28e22640 1 -- 172.21.15.53:0/1619998599 >> 172.21.15.53:0/1619998599 conn(0x7feb241011a0 msgr2=0x7feb241022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:29.025 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.447+0000 7feb28e22640 1 -- 172.21.15.53:0/1619998599 shutdown_connections 2024-09-19T06:09:29.025 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:28.447+0000 7feb28e22640 1 -- 172.21.15.53:0/1619998599 wait complete. 2024-09-19T06:09:29.025 INFO:teuthology.orchestra.run.smithi053.stdout:Generating new minimal ceph.conf... 2024-09-19T06:09:30.061 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:29 smithi053 ceph-mon[27605]: from='client.? 172.21.15.53:0/1495969966' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:09:30.068 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.422+0000 7f6f628b1640 1 Processor -- start 2024-09-19T06:09:30.068 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.422+0000 7f6f628b1640 1 -- start start 2024-09-19T06:09:30.068 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.423+0000 7f6f628b1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f5c107540 0x7f6f5c109930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:30.068 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.423+0000 7f6f628b1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c07a730 con 0x7f6f5c107540 2024-09-19T06:09:30.068 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.423+0000 7f6f5bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f5c107540 0x7f6f5c109930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:30.068 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.423+0000 7f6f5bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f5c107540 0x7f6f5c109930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36680/0 (socket says 172.21.15.53:36680) 2024-09-19T06:09:30.068 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.423+0000 7f6f5bfff640 1 -- 172.21.15.53:0/2486151562 learned_addr learned my addr 172.21.15.53:0/2486151562 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:09:30.068 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.423+0000 7f6f5bfff640 1 -- 172.21.15.53:0/2486151562 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f5c109e70 con 0x7f6f5c107540 2024-09-19T06:09:30.068 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.424+0000 7f6f5bfff640 1 --2- 172.21.15.53:0/2486151562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f5c107540 0x7f6f5c109930 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c009820 tx=0x7f6f4c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc44eaf4503f2fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:30.069 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.424+0000 7f6f5affd640 1 -- 172.21.15.53:0/2486151562 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f6f4c037070 con 0x7f6f5c107540 2024-09-19T06:09:30.069 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.424+0000 7f6f5affd640 1 -- 172.21.15.53:0/2486151562 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6f4c02ad40 con 0x7f6f5c107540 2024-09-19T06:09:30.069 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.424+0000 7f6f5affd640 1 -- 172.21.15.53:0/2486151562 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f6f4c030670 con 0x7f6f5c107540 2024-09-19T06:09:30.069 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.425+0000 7f6f628b1640 1 -- 172.21.15.53:0/2486151562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f5c107540 msgr2=0x7f6f5c109930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:30.069 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.425+0000 7f6f628b1640 1 --2- 172.21.15.53:0/2486151562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f5c107540 0x7f6f5c109930 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c009820 tx=0x7f6f4c02a270 comp rx=0 tx=0).stop 2024-09-19T06:09:30.069 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.425+0000 7f6f628b1640 1 -- 172.21.15.53:0/2486151562 shutdown_connections 2024-09-19T06:09:30.069 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.425+0000 7f6f628b1640 1 --2- 172.21.15.53:0/2486151562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f5c107540 0x7f6f5c109930 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:30.069 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.425+0000 7f6f628b1640 1 -- 172.21.15.53:0/2486151562 >> 172.21.15.53:0/2486151562 conn(0x7f6f5c1011a0 msgr2=0x7f6f5c1035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:30.069 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.425+0000 7f6f628b1640 1 -- 172.21.15.53:0/2486151562 shutdown_connections 2024-09-19T06:09:30.069 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.425+0000 7f6f628b1640 1 -- 172.21.15.53:0/2486151562 wait complete. 2024-09-19T06:09:30.069 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.425+0000 7f6f628b1640 1 Processor -- start 2024-09-19T06:09:30.069 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.425+0000 7f6f628b1640 1 -- start start 2024-09-19T06:09:30.069 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.426+0000 7f6f628b1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f5c107540 0x7f6f5c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:30.070 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.426+0000 7f6f628b1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c10aeb0 con 0x7f6f5c107540 2024-09-19T06:09:30.070 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.426+0000 7f6f5bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f5c107540 0x7f6f5c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:30.070 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.426+0000 7f6f5bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f5c107540 0x7f6f5c19aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36696/0 (socket says 172.21.15.53:36696) 2024-09-19T06:09:30.070 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.426+0000 7f6f5bfff640 1 -- 172.21.15.53:0/1495969966 learned_addr learned my addr 172.21.15.53:0/1495969966 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:09:30.070 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.426+0000 7f6f5bfff640 1 -- 172.21.15.53:0/1495969966 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f5c19af70 con 0x7f6f5c107540 2024-09-19T06:09:30.070 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.426+0000 7f6f5bfff640 1 --2- 172.21.15.53:0/1495969966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f5c107540 0x7f6f5c19aa30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c000c00 tx=0x7f6f4c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:30.070 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.427+0000 7f6f597fa640 1 -- 172.21.15.53:0/1495969966 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f6f4c043070 con 0x7f6f5c107540 2024-09-19T06:09:30.070 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.427+0000 7f6f597fa640 1 -- 172.21.15.53:0/1495969966 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6f4c03e480 con 0x7f6f5c107540 2024-09-19T06:09:30.070 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.427+0000 7f6f628b1640 1 -- 172.21.15.53:0/1495969966 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f5c19b200 con 0x7f6f5c107540 2024-09-19T06:09:30.070 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.427+0000 7f6f597fa640 1 -- 172.21.15.53:0/1495969966 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f6f4c03c530 con 0x7f6f5c107540 2024-09-19T06:09:30.070 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.427+0000 7f6f628b1640 1 -- 172.21.15.53:0/1495969966 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f5c19def0 con 0x7f6f5c107540 2024-09-19T06:09:30.070 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.427+0000 7f6f597fa640 1 -- 172.21.15.53:0/1495969966 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6f4c046070 con 0x7f6f5c107540 2024-09-19T06:09:30.070 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.427+0000 7f6f597fa640 1 -- 172.21.15.53:0/1495969966 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6f4c0489e0 con 0x7f6f5c107540 2024-09-19T06:09:30.071 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.428+0000 7f6f628b1640 1 -- 172.21.15.53:0/1495969966 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f5c1079a0 con 0x7f6f5c107540 2024-09-19T06:09:30.071 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.430+0000 7f6f597fa640 1 -- 172.21.15.53:0/1495969966 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f6f4c03ea10 con 0x7f6f5c107540 2024-09-19T06:09:30.071 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.489+0000 7f6f628b1640 1 -- 172.21.15.53:0/1495969966 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f6f5c19e460 con 0x7f6f5c107540 2024-09-19T06:09:30.071 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.490+0000 7f6f597fa640 1 -- 172.21.15.53:0/1495969966 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f6f4c03e630 con 0x7f6f5c107540 2024-09-19T06:09:30.071 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.491+0000 7f6f628b1640 1 -- 172.21.15.53:0/1495969966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f5c107540 msgr2=0x7f6f5c19aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:30.071 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.491+0000 7f6f628b1640 1 --2- 172.21.15.53:0/1495969966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f5c107540 0x7f6f5c19aa30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c000c00 tx=0x7f6f4c032670 comp rx=0 tx=0).stop 2024-09-19T06:09:30.071 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.491+0000 7f6f628b1640 1 -- 172.21.15.53:0/1495969966 shutdown_connections 2024-09-19T06:09:30.071 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.491+0000 7f6f628b1640 1 --2- 172.21.15.53:0/1495969966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f5c107540 0x7f6f5c19aa30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:30.071 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.491+0000 7f6f628b1640 1 -- 172.21.15.53:0/1495969966 >> 172.21.15.53:0/1495969966 conn(0x7f6f5c1011a0 msgr2=0x7f6f5c101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:30.071 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.491+0000 7f6f628b1640 1 -- 172.21.15.53:0/1495969966 shutdown_connections 2024-09-19T06:09:30.071 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:29.491+0000 7f6f628b1640 1 -- 172.21.15.53:0/1495969966 wait complete. 2024-09-19T06:09:30.071 INFO:teuthology.orchestra.run.smithi053.stdout:Restarting the monitor... 2024-09-19T06:09:30.349 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:30 smithi053 systemd[1]: Stopping Ceph mon.smithi053 for 9f2876d0-764d-11ef-bceb-c7b262605968... 2024-09-19T06:09:30.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:30 smithi053 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-mon-smithi053[27601]: 2024-09-19T06:09:30.346+0000 7fa565b7d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi053 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:09:30.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:30 smithi053 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-mon-smithi053[27601]: 2024-09-19T06:09:30.346+0000 7fa565b7d640 -1 mon.smithi053@0(leader) e1 *** Got Signal Terminated *** 2024-09-19T06:09:30.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:30 smithi053 podman[27848]: 2024-09-19 06:09:30.490733808 +0000 UTC m=+0.259661687 container died d5b2f70151d8ddc7cdf78b1e2a71db6161a1056665776c534832f8da218b2454 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-mon-smithi053, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:09:31.099 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:30 smithi053 podman[27848]: 2024-09-19 06:09:30.838229558 +0000 UTC m=+0.607157435 container cleanup d5b2f70151d8ddc7cdf78b1e2a71db6161a1056665776c534832f8da218b2454 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-mon-smithi053, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-19T06:09:31.099 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:30 smithi053 bash[27848]: ceph-9f2876d0-764d-11ef-bceb-c7b262605968-mon-smithi053 2024-09-19T06:09:31.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 podman[27860]: 2024-09-19 06:09:31.09752843 +0000 UTC m=+0.601467308 container remove d5b2f70151d8ddc7cdf78b1e2a71db6161a1056665776c534832f8da218b2454 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-mon-smithi053, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:09:31.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 systemd[1]: ceph-9f2876d0-764d-11ef-bceb-c7b262605968@mon.smithi053.service: Deactivated successfully. 2024-09-19T06:09:31.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 systemd[1]: Stopped Ceph mon.smithi053 for 9f2876d0-764d-11ef-bceb-c7b262605968. 2024-09-19T06:09:31.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 systemd[1]: Starting Ceph mon.smithi053 for 9f2876d0-764d-11ef-bceb-c7b262605968... 2024-09-19T06:09:31.711 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 podman[27949]: 2024-09-19 06:09:31.448394699 +0000 UTC m=+0.091231349 container create 3a3c39f8318dc4298a0aeedccb7c17526d4a93ddf388f71f17f5a78fcf5e7433 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-mon-smithi053, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-19T06:09:31.711 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 podman[27949]: 2024-09-19 06:09:31.373392841 +0000 UTC m=+0.016229448 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:09:31.711 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 podman[27949]: 2024-09-19 06:09:31.683936623 +0000 UTC m=+0.326773233 container init 3a3c39f8318dc4298a0aeedccb7c17526d4a93ddf388f71f17f5a78fcf5e7433 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-mon-smithi053, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0) 2024-09-19T06:09:31.711 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 podman[27949]: 2024-09-19 06:09:31.687402426 +0000 UTC m=+0.330239034 container start 3a3c39f8318dc4298a0aeedccb7c17526d4a93ddf388f71f17f5a78fcf5e7433 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-mon-smithi053, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-09-19T06:09:31.773 INFO:teuthology.orchestra.run.smithi053.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-19T06:09:32.054 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T06:09:32.055 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T06:09:32.055 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: pidfile_write: ignore empty --pid-file 2024-09-19T06:09:32.055 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: load: jerasure load: lrc 2024-09-19T06:09:32.055 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: RocksDB version: 7.9.2 2024-09-19T06:09:32.055 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Git sha 0 2024-09-19T06:09:32.056 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T06:09:32.056 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: DB SUMMARY 2024-09-19T06:09:32.056 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: DB Session ID: DR7JF9ZOBFFEQDAKBKBK 2024-09-19T06:09:32.056 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: CURRENT file: CURRENT 2024-09-19T06:09:32.056 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T06:09:32.056 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-19T06:09:32.056 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi053/store.db dir, Total Num: 1, files: 000008.sst 2024-09-19T06:09:32.056 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi053/store.db: 000009.log size: 98261 ; 2024-09-19T06:09:32.057 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.error_if_exists: 0 2024-09-19T06:09:32.057 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.create_if_missing: 0 2024-09-19T06:09:32.057 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.paranoid_checks: 1 2024-09-19T06:09:32.057 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T06:09:32.057 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T06:09:32.057 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T06:09:32.057 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.env: 0x564a69e74940 2024-09-19T06:09:32.057 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T06:09:32.057 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.info_log: 0x564a6a8038c0 2024-09-19T06:09:32.057 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T06:09:32.057 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.statistics: (nil) 2024-09-19T06:09:32.057 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.use_fsync: 0 2024-09-19T06:09:32.057 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.max_log_file_size: 0 2024-09-19T06:09:32.057 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T06:09:32.057 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T06:09:32.058 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T06:09:32.058 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T06:09:32.058 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.allow_fallocate: 1 2024-09-19T06:09:32.058 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T06:09:32.058 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T06:09:32.058 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.use_direct_reads: 0 2024-09-19T06:09:32.058 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T06:09:32.058 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T06:09:32.058 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.db_log_dir: 2024-09-19T06:09:32.058 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.wal_dir: 2024-09-19T06:09:32.058 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T06:09:32.058 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T06:09:32.058 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T06:09:32.058 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T06:09:32.059 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T06:09:32.059 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T06:09:32.059 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T06:09:32.059 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T06:09:32.059 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.write_buffer_manager: 0x564a6a812dc0 2024-09-19T06:09:32.059 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T06:09:32.059 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T06:09:32.059 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T06:09:32.059 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T06:09:32.059 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T06:09:32.059 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T06:09:32.059 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T06:09:32.059 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T06:09:32.059 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.unordered_write: 0 2024-09-19T06:09:32.059 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T06:09:32.060 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T06:09:32.060 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T06:09:32.060 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T06:09:32.060 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.row_cache: None 2024-09-19T06:09:32.060 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.wal_filter: None 2024-09-19T06:09:32.060 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T06:09:32.060 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T06:09:32.060 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.two_write_queues: 0 2024-09-19T06:09:32.060 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T06:09:32.060 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.wal_compression: 0 2024-09-19T06:09:32.061 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.atomic_flush: 0 2024-09-19T06:09:32.061 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T06:09:32.061 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T06:09:32.061 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T06:09:32.061 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.log_readahead_size: 0 2024-09-19T06:09:32.061 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T06:09:32.061 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T06:09:32.061 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T06:09:32.061 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T06:09:32.061 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T06:09:32.061 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T06:09:32.061 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T06:09:32.061 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.max_background_jobs: 2 2024-09-19T06:09:32.062 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.max_background_compactions: -1 2024-09-19T06:09:32.062 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.max_subcompactions: 1 2024-09-19T06:09:32.062 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T06:09:32.062 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T06:09:32.062 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T06:09:32.062 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T06:09:32.062 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T06:09:32.062 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T06:09:32.062 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T06:09:32.062 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T06:09:32.062 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.max_open_files: -1 2024-09-19T06:09:32.062 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T06:09:32.062 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T06:09:32.062 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T06:09:32.063 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T06:09:32.063 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.max_background_flushes: -1 2024-09-19T06:09:32.063 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Compression algorithms supported: 2024-09-19T06:09:32.063 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: kZSTD supported: 0 2024-09-19T06:09:32.063 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: kXpressCompression supported: 0 2024-09-19T06:09:32.063 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: kBZip2Compression supported: 0 2024-09-19T06:09:32.063 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T06:09:32.063 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: kLZ4Compression supported: 1 2024-09-19T06:09:32.063 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: kZlibCompression supported: 1 2024-09-19T06:09:32.063 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T06:09:32.063 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: kSnappyCompression supported: 1 2024-09-19T06:09:32.063 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T06:09:32.063 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T06:09:32.063 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi053/store.db/MANIFEST-000010 2024-09-19T06:09:32.064 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T06:09:32.064 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T06:09:32.064 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.merge_operator: 2024-09-19T06:09:32.064 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.compaction_filter: None 2024-09-19T06:09:32.064 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T06:09:32.064 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T06:09:32.064 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T06:09:32.064 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T06:09:32.064 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x564a6a803520) 2024-09-19T06:09:32.064 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: cache_index_and_filter_blocks: 1 2024-09-19T06:09:32.064 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T06:09:32.064 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T06:09:32.064 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: pin_top_level_index_and_filter: 1 2024-09-19T06:09:32.064 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: index_type: 0 2024-09-19T06:09:32.065 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: data_block_index_type: 0 2024-09-19T06:09:32.065 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: index_shortening: 1 2024-09-19T06:09:32.065 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T06:09:32.065 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: checksum: 4 2024-09-19T06:09:32.065 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: no_block_cache: 0 2024-09-19T06:09:32.065 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: block_cache: 0x564a6a7f71f0 2024-09-19T06:09:32.065 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: block_cache_name: BinnedLRUCache 2024-09-19T06:09:32.065 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: block_cache_options: 2024-09-19T06:09:32.065 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: capacity : 536870912 2024-09-19T06:09:32.065 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: num_shard_bits : 4 2024-09-19T06:09:32.065 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: strict_capacity_limit : 0 2024-09-19T06:09:32.065 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: high_pri_pool_ratio: 0.000 2024-09-19T06:09:32.065 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: block_cache_compressed: (nil) 2024-09-19T06:09:32.065 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: persistent_cache: (nil) 2024-09-19T06:09:32.066 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: block_size: 4096 2024-09-19T06:09:32.066 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: block_size_deviation: 10 2024-09-19T06:09:32.066 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: block_restart_interval: 16 2024-09-19T06:09:32.066 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: index_block_restart_interval: 1 2024-09-19T06:09:32.066 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: metadata_block_size: 4096 2024-09-19T06:09:32.066 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: partition_filters: 0 2024-09-19T06:09:32.066 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: use_delta_encoding: 1 2024-09-19T06:09:32.066 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: filter_policy: bloomfilter 2024-09-19T06:09:32.066 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: whole_key_filtering: 1 2024-09-19T06:09:32.066 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: verify_compression: 0 2024-09-19T06:09:32.066 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: read_amp_bytes_per_bit: 0 2024-09-19T06:09:32.066 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: format_version: 5 2024-09-19T06:09:32.066 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: enable_index_compression: 1 2024-09-19T06:09:32.066 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: block_align: 0 2024-09-19T06:09:32.067 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: max_auto_readahead_size: 262144 2024-09-19T06:09:32.067 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: prepopulate_block_cache: 0 2024-09-19T06:09:32.067 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: initial_auto_readahead_size: 8192 2024-09-19T06:09:32.067 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T06:09:32.067 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T06:09:32.067 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T06:09:32.067 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.compression: NoCompression 2024-09-19T06:09:32.067 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T06:09:32.067 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T06:09:32.067 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T06:09:32.067 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.num_levels: 7 2024-09-19T06:09:32.067 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T06:09:32.067 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T06:09:32.067 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T06:09:32.068 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T06:09:32.068 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T06:09:32.068 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T06:09:32.068 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T06:09:32.068 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T06:09:32.069 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T06:09:32.069 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T06:09:32.069 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T06:09:32.069 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T06:09:32.069 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T06:09:32.069 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T06:09:32.069 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T06:09:32.069 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T06:09:32.069 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T06:09:32.069 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T06:09:32.069 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T06:09:32.069 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T06:09:32.070 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T06:09:32.070 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T06:09:32.070 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T06:09:32.070 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T06:09:32.070 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T06:09:32.070 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T06:09:32.070 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T06:09:32.070 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T06:09:32.070 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T06:09:32.070 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T06:09:32.070 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T06:09:32.071 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T06:09:32.071 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T06:09:32.071 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T06:09:32.071 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T06:09:32.071 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T06:09:32.071 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T06:09:32.071 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T06:09:32.071 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T06:09:32.071 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T06:09:32.071 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T06:09:32.071 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T06:09:32.071 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T06:09:32.072 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T06:09:32.072 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T06:09:32.072 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T06:09:32.072 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T06:09:32.072 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T06:09:32.072 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T06:09:32.072 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T06:09:32.072 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T06:09:32.072 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T06:09:32.072 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T06:09:32.072 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T06:09:32.072 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.inplace_update_support: 0 2024-09-19T06:09:32.073 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T06:09:32.073 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T06:09:32.073 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T06:09:32.073 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T06:09:32.073 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.bloom_locality: 0 2024-09-19T06:09:32.073 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.max_successive_merges: 0 2024-09-19T06:09:32.073 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T06:09:32.073 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T06:09:32.073 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T06:09:32.073 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T06:09:32.073 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.ttl: 2592000 2024-09-19T06:09:32.073 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T06:09:32.073 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T06:09:32.073 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T06:09:32.073 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.enable_blob_files: false 2024-09-19T06:09:32.074 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.min_blob_size: 0 2024-09-19T06:09:32.074 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T06:09:32.074 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T06:09:32.074 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T06:09:32.074 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T06:09:32.074 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T06:09:32.074 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T06:09:32.074 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T06:09:32.074 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T06:09:32.074 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi053/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-19T06:09:32.074 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-19T06:09:32.074 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ccd33905-a59f-485b-8957-769838f67e34 2024-09-19T06:09:32.074 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726726171715764, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-19T06:09:32.074 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-19T06:09:32.074 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726726171716789, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94047, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 296, "table_properties": {"data_size": 92094, "index_size": 273, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 12360, "raw_average_key_size": 48, "raw_value_size": 85263, "raw_average_value_size": 334, "num_data_blocks": 12, "num_entries": 255, "num_filter_entries": 255, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726726171, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ccd33905-a59f-485b-8957-769838f67e34", "db_session_id": "DR7JF9ZOBFFEQDAKBKBK", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-19T06:09:32.074 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726726171716894, "job": 1, "event": "recovery_finished"} 2024-09-19T06:09:32.075 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-19T06:09:32.075 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi053/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T06:09:32.075 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x564a6a824e00 2024-09-19T06:09:32.075 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: DB pointer 0x564a6a920000 2024-09-19T06:09:32.075 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T06:09:32.075 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T06:09:32.075 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: ** DB Stats ** 2024-09-19T06:09:32.075 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T06:09:32.075 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T06:09:32.075 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T06:09:32.075 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T06:09:32.075 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T06:09:32.075 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T06:09:32.075 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T06:09:32.075 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: 2024-09-19T06:09:32.076 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: ** Compaction Stats [default] ** 2024-09-19T06:09:32.076 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T06:09:32.076 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T06:09:32.076 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: L0 2/0 93.74 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 134.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T06:09:32.076 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Sum 2/0 93.74 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 134.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T06:09:32.076 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 134.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T06:09:32.076 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: 2024-09-19T06:09:32.076 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: ** Compaction Stats [default] ** 2024-09-19T06:09:32.076 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T06:09:32.076 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T06:09:32.076 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 134.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T06:09:32.076 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: 2024-09-19T06:09:32.076 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T06:09:32.076 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: 2024-09-19T06:09:32.077 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T06:09:32.077 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T06:09:32.077 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T06:09:32.077 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T06:09:32.077 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T06:09:32.077 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T06:09:32.077 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Cumulative compaction: 0.00 GB write, 27.86 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T06:09:32.077 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Interval compaction: 0.00 GB write, 27.86 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T06:09:32.077 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T06:09:32.077 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Block cache BinnedLRUCache@0x564a6a7f71f0#2 capacity: 512.00 MB usage: 1.30 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-19T06:09:32.077 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.83 KB,0.000157952%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-19T06:09:32.077 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: 2024-09-19T06:09:32.077 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T06:09:32.077 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: starting mon.smithi053 rank 0 at public addrs [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] at bind addrs [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi053 fsid 9f2876d0-764d-11ef-bceb-c7b262605968 2024-09-19T06:09:32.078 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: mon.smithi053@-1(???) e1 preinit fsid 9f2876d0-764d-11ef-bceb-c7b262605968 2024-09-19T06:09:32.078 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: mon.smithi053@-1(???).mds e1 new map 2024-09-19T06:09:32.078 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: mon.smithi053@-1(???).mds e1 print_map 2024-09-19T06:09:32.078 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: e1 2024-09-19T06:09:32.078 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: btime 2024-09-19T06:09:26:797704+0000 2024-09-19T06:09:32.078 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T06:09:32.078 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T06:09:32.078 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: legacy client fscid: -1 2024-09-19T06:09:32.078 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: 2024-09-19T06:09:32.078 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: No filesystems configured 2024-09-19T06:09:32.078 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: mon.smithi053@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T06:09:32.078 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: mon.smithi053@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T06:09:32.078 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: mon.smithi053@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T06:09:32.078 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: mon.smithi053@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T06:09:32.078 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: mon.smithi053@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-19T06:09:32.078 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 bash[27949]: 3a3c39f8318dc4298a0aeedccb7c17526d4a93ddf388f71f17f5a78fcf5e7433 2024-09-19T06:09:32.079 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 systemd[1]: Started Ceph mon.smithi053 for 9f2876d0-764d-11ef-bceb-c7b262605968. 2024-09-19T06:09:32.079 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: mon.smithi053 is new leader, mons smithi053 in quorum (ranks 0) 2024-09-19T06:09:32.079 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: monmap epoch 1 2024-09-19T06:09:32.079 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: fsid 9f2876d0-764d-11ef-bceb-c7b262605968 2024-09-19T06:09:32.079 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: last_changed 2024-09-19T06:09:23.791881+0000 2024-09-19T06:09:32.079 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: created 2024-09-19T06:09:23.791881+0000 2024-09-19T06:09:32.079 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: min_mon_release 19 (squid) 2024-09-19T06:09:32.079 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: election_strategy: 1 2024-09-19T06:09:32.079 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: 0: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.smithi053 2024-09-19T06:09:32.079 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: fsmap 2024-09-19T06:09:32.079 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T06:09:32.079 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:31 smithi053 ceph-mon[27968]: mgrmap e1: no daemons active 2024-09-19T06:09:32.841 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.246+0000 7faca3bfc640 1 Processor -- start 2024-09-19T06:09:32.841 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.247+0000 7faca3bfc640 1 -- start start 2024-09-19T06:09:32.841 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.247+0000 7faca3bfc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac9c107540 0x7fac9c109930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:32.841 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.247+0000 7faca3bfc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c07a730 con 0x7fac9c107540 2024-09-19T06:09:32.841 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.247+0000 7faca1998640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac9c107540 0x7fac9c109930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:32.841 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.247+0000 7faca1998640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac9c107540 0x7fac9c109930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39780/0 (socket says 172.21.15.53:39780) 2024-09-19T06:09:32.841 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.247+0000 7faca1998640 1 -- 172.21.15.53:0/80836004 learned_addr learned my addr 172.21.15.53:0/80836004 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:09:32.841 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.247+0000 7faca1998640 1 -- 172.21.15.53:0/80836004 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c109e70 con 0x7fac9c107540 2024-09-19T06:09:32.842 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.249+0000 7faca1998640 1 --2- 172.21.15.53:0/80836004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac9c107540 0x7fac9c109930 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fac98009820 tx=0x7fac9802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7bfdbf6ab08789b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:32.842 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.249+0000 7faca0996640 1 -- 172.21.15.53:0/80836004 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fac98037070 con 0x7fac9c107540 2024-09-19T06:09:32.842 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.249+0000 7faca0996640 1 -- 172.21.15.53:0/80836004 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fac9802ad40 con 0x7fac9c107540 2024-09-19T06:09:32.842 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.249+0000 7faca0996640 1 -- 172.21.15.53:0/80836004 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fac98030670 con 0x7fac9c107540 2024-09-19T06:09:32.842 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.250+0000 7faca3bfc640 1 -- 172.21.15.53:0/80836004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac9c107540 msgr2=0x7fac9c109930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:32.842 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.250+0000 7faca3bfc640 1 --2- 172.21.15.53:0/80836004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac9c107540 0x7fac9c109930 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fac98009820 tx=0x7fac9802a270 comp rx=0 tx=0).stop 2024-09-19T06:09:32.842 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.250+0000 7faca3bfc640 1 -- 172.21.15.53:0/80836004 shutdown_connections 2024-09-19T06:09:32.842 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.250+0000 7faca3bfc640 1 --2- 172.21.15.53:0/80836004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac9c107540 0x7fac9c109930 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:32.842 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.250+0000 7faca3bfc640 1 -- 172.21.15.53:0/80836004 >> 172.21.15.53:0/80836004 conn(0x7fac9c1011a0 msgr2=0x7fac9c1035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:32.842 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.250+0000 7faca3bfc640 1 -- 172.21.15.53:0/80836004 shutdown_connections 2024-09-19T06:09:32.842 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.250+0000 7faca3bfc640 1 -- 172.21.15.53:0/80836004 wait complete. 2024-09-19T06:09:32.842 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.251+0000 7faca3bfc640 1 Processor -- start 2024-09-19T06:09:32.842 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.251+0000 7faca3bfc640 1 -- start start 2024-09-19T06:09:32.842 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.251+0000 7faca3bfc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac9c107540 0x7fac9c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:32.843 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.251+0000 7faca3bfc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c10aeb0 con 0x7fac9c107540 2024-09-19T06:09:32.843 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.251+0000 7faca1998640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac9c107540 0x7fac9c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:32.843 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.251+0000 7faca1998640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac9c107540 0x7fac9c19aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39784/0 (socket says 172.21.15.53:39784) 2024-09-19T06:09:32.843 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.251+0000 7faca1998640 1 -- 172.21.15.53:0/2248267420 learned_addr learned my addr 172.21.15.53:0/2248267420 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:09:32.843 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.251+0000 7faca1998640 1 -- 172.21.15.53:0/2248267420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c19af70 con 0x7fac9c107540 2024-09-19T06:09:32.843 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.251+0000 7faca1998640 1 --2- 172.21.15.53:0/2248267420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac9c107540 0x7fac9c19aa30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fac98000c00 tx=0x7fac98032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:32.843 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.252+0000 7fac92ffd640 1 -- 172.21.15.53:0/2248267420 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fac98043070 con 0x7fac9c107540 2024-09-19T06:09:32.843 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.252+0000 7fac92ffd640 1 -- 172.21.15.53:0/2248267420 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fac9803e480 con 0x7fac9c107540 2024-09-19T06:09:32.843 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.252+0000 7faca3bfc640 1 -- 172.21.15.53:0/2248267420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac9c19b200 con 0x7fac9c107540 2024-09-19T06:09:32.843 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.252+0000 7fac92ffd640 1 -- 172.21.15.53:0/2248267420 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fac9803c530 con 0x7fac9c107540 2024-09-19T06:09:32.843 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.252+0000 7faca3bfc640 1 -- 172.21.15.53:0/2248267420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac9c19def0 con 0x7fac9c107540 2024-09-19T06:09:32.843 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.252+0000 7fac92ffd640 1 -- 172.21.15.53:0/2248267420 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fac98046070 con 0x7fac9c107540 2024-09-19T06:09:32.843 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.252+0000 7fac92ffd640 1 -- 172.21.15.53:0/2248267420 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fac980489e0 con 0x7fac9c107540 2024-09-19T06:09:32.843 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.253+0000 7faca3bfc640 1 -- 172.21.15.53:0/2248267420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac9c1079a0 con 0x7fac9c107540 2024-09-19T06:09:32.843 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.255+0000 7fac92ffd640 1 -- 172.21.15.53:0/2248267420 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fac9803ea10 con 0x7fac9c107540 2024-09-19T06:09:32.844 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.314+0000 7faca3bfc640 1 -- 172.21.15.53:0/2248267420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fac9c19e4d0 con 0x7fac9c107540 2024-09-19T06:09:32.844 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.316+0000 7fac92ffd640 1 -- 172.21.15.53:0/2248267420 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fac9803e630 con 0x7fac9c107540 2024-09-19T06:09:32.844 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.317+0000 7faca3bfc640 1 -- 172.21.15.53:0/2248267420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac9c107540 msgr2=0x7fac9c19aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:32.844 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.317+0000 7faca3bfc640 1 --2- 172.21.15.53:0/2248267420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac9c107540 0x7fac9c19aa30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fac98000c00 tx=0x7fac98032670 comp rx=0 tx=0).stop 2024-09-19T06:09:32.844 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.317+0000 7faca3bfc640 1 -- 172.21.15.53:0/2248267420 shutdown_connections 2024-09-19T06:09:32.844 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.317+0000 7faca3bfc640 1 --2- 172.21.15.53:0/2248267420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac9c107540 0x7fac9c19aa30 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:32.844 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.317+0000 7faca3bfc640 1 -- 172.21.15.53:0/2248267420 >> 172.21.15.53:0/2248267420 conn(0x7fac9c1011a0 msgr2=0x7fac9c101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:32.844 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.317+0000 7faca3bfc640 1 -- 172.21.15.53:0/2248267420 shutdown_connections 2024-09-19T06:09:32.844 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:32.317+0000 7faca3bfc640 1 -- 172.21.15.53:0/2248267420 wait complete. 2024-09-19T06:09:32.844 INFO:teuthology.orchestra.run.smithi053.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-19T06:09:32.849 INFO:teuthology.orchestra.run.smithi053.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-19T06:09:32.849 INFO:teuthology.orchestra.run.smithi053.stdout:Creating mgr... 2024-09-19T06:09:32.850 INFO:teuthology.orchestra.run.smithi053.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-19T06:09:32.851 INFO:teuthology.orchestra.run.smithi053.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-19T06:09:32.851 INFO:teuthology.orchestra.run.smithi053.stdout:Verifying port 0.0.0.0:8443 ... 2024-09-19T06:09:33.077 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9f2876d0-764d-11ef-bceb-c7b262605968@mgr.smithi053.trakvg 2024-09-19T06:09:33.077 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9f2876d0-764d-11ef-bceb-c7b262605968@mgr.smithi053.trakvg.service: Unit ceph-9f2876d0-764d-11ef-bceb-c7b262605968@mgr.smithi053.trakvg.service not loaded. 2024-09-19T06:09:33.237 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9f2876d0-764d-11ef-bceb-c7b262605968.target.wants/ceph-9f2876d0-764d-11ef-bceb-c7b262605968@mgr.smithi053.trakvg.service → /etc/systemd/system/ceph-9f2876d0-764d-11ef-bceb-c7b262605968@.service. 2024-09-19T06:09:33.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:33 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/2248267420' entity='client.admin' 2024-09-19T06:09:33.803 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T06:09:33.803 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout disabled 2024-09-19T06:09:33.811 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T06:09:33.811 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout inactive 2024-09-19T06:09:33.811 INFO:teuthology.orchestra.run.smithi053.stdout:firewalld.service is not enabled 2024-09-19T06:09:33.811 INFO:teuthology.orchestra.run.smithi053.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T06:09:33.820 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T06:09:33.820 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout disabled 2024-09-19T06:09:33.828 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T06:09:33.829 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout inactive 2024-09-19T06:09:33.829 INFO:teuthology.orchestra.run.smithi053.stdout:firewalld.service is not enabled 2024-09-19T06:09:33.829 INFO:teuthology.orchestra.run.smithi053.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-09-19T06:09:33.829 INFO:teuthology.orchestra.run.smithi053.stdout:Waiting for mgr to start... 2024-09-19T06:09:33.829 INFO:teuthology.orchestra.run.smithi053.stdout:Waiting for mgr... 2024-09-19T06:09:34.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:34 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/3562707432' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T06:09:34.872 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2024-09-19T06:09:34.884 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout { 2024-09-19T06:09:34.884 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "fsid": "9f2876d0-764d-11ef-bceb-c7b262605968", 2024-09-19T06:09:34.884 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T06:09:34.884 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T06:09:34.884 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T06:09:34.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T06:09:34.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:09:34.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T06:09:34.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T06:09:34.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 0 2024-09-19T06:09:34.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2024-09-19T06:09:34.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T06:09:34.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "smithi053" 2024-09-19T06:09:34.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2024-09-19T06:09:34.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-19T06:09:34.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T06:09:34.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:09:34.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T06:09:34.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T06:09:34.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:09:34.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T06:09:34.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:09:34.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T06:09:34.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T06:09:34.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T06:09:34.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T06:09:34.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T06:09:34.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T06:09:34.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:09:34.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T06:09:34.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T06:09:34.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T06:09:34.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T06:09:34.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T06:09:34.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T06:09:34.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T06:09:34.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T06:09:34.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T06:09:34.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:09:34.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T06:09:34.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:09:34.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T06:09:26:797704+0000", 2024-09-19T06:09:34.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T06:09:34.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T06:09:34.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:09:34.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T06:09:34.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T06:09:34.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T06:09:34.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T06:09:34.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T06:09:34.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T06:09:34.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T06:09:34.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2024-09-19T06:09:34.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T06:09:34.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:09:34.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T06:09:34.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:09:34.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T06:09:26.798327+0000", 2024-09-19T06:09:34.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T06:09:34.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:09:34.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T06:09:34.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout } 2024-09-19T06:09:34.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.237+0000 7f3a3c5f8640 1 Processor -- start 2024-09-19T06:09:34.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.237+0000 7f3a3c5f8640 1 -- start start 2024-09-19T06:09:34.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.237+0000 7f3a3c5f8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a341059a0 0x7f3a34105da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:34.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.238+0000 7f3a3c5f8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3a34106370 con 0x7f3a341059a0 2024-09-19T06:09:34.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.238+0000 7f3a3a394640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a341059a0 0x7f3a34105da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:34.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.238+0000 7f3a3a394640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a341059a0 0x7f3a34105da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39818/0 (socket says 172.21.15.53:39818) 2024-09-19T06:09:34.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.238+0000 7f3a3a394640 1 -- 172.21.15.53:0/3508232624 learned_addr learned my addr 172.21.15.53:0/3508232624 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:09:34.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.238+0000 7f3a3a394640 1 -- 172.21.15.53:0/3508232624 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a34106ba0 con 0x7f3a341059a0 2024-09-19T06:09:34.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.239+0000 7f3a3a394640 1 --2- 172.21.15.53:0/3508232624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a341059a0 0x7f3a34105da0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3a30009820 tx=0x7f3a3002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f9e691a93c70fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:34.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.239+0000 7f3a39392640 1 -- 172.21.15.53:0/3508232624 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f3a30037070 con 0x7f3a341059a0 2024-09-19T06:09:34.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.239+0000 7f3a39392640 1 -- 172.21.15.53:0/3508232624 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3a3002ad40 con 0x7f3a341059a0 2024-09-19T06:09:34.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.239+0000 7f3a39392640 1 -- 172.21.15.53:0/3508232624 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f3a30030670 con 0x7f3a341059a0 2024-09-19T06:09:34.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.240+0000 7f3a3c5f8640 1 -- 172.21.15.53:0/3508232624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a341059a0 msgr2=0x7f3a34105da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:34.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.240+0000 7f3a3c5f8640 1 --2- 172.21.15.53:0/3508232624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a341059a0 0x7f3a34105da0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3a30009820 tx=0x7f3a3002a270 comp rx=0 tx=0).stop 2024-09-19T06:09:34.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.240+0000 7f3a3c5f8640 1 -- 172.21.15.53:0/3508232624 shutdown_connections 2024-09-19T06:09:34.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.240+0000 7f3a3c5f8640 1 --2- 172.21.15.53:0/3508232624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a341059a0 0x7f3a34105da0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.240+0000 7f3a3c5f8640 1 -- 172.21.15.53:0/3508232624 >> 172.21.15.53:0/3508232624 conn(0x7f3a34101190 msgr2=0x7f3a341035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:34.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.240+0000 7f3a3c5f8640 1 -- 172.21.15.53:0/3508232624 shutdown_connections 2024-09-19T06:09:34.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.240+0000 7f3a3c5f8640 1 -- 172.21.15.53:0/3508232624 wait complete. 2024-09-19T06:09:34.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.241+0000 7f3a3c5f8640 1 Processor -- start 2024-09-19T06:09:34.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.241+0000 7f3a3c5f8640 1 -- start start 2024-09-19T06:09:34.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.241+0000 7f3a3c5f8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a341059a0 0x7f3a34199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:34.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.241+0000 7f3a3c5f8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3a34109f50 con 0x7f3a341059a0 2024-09-19T06:09:34.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.241+0000 7f3a3a394640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a341059a0 0x7f3a34199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:34.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.241+0000 7f3a3a394640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a341059a0 0x7f3a34199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39826/0 (socket says 172.21.15.53:39826) 2024-09-19T06:09:34.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.241+0000 7f3a3a394640 1 -- 172.21.15.53:0/3562707432 learned_addr learned my addr 172.21.15.53:0/3562707432 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:09:34.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.241+0000 7f3a3a394640 1 -- 172.21.15.53:0/3562707432 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a34199ea0 con 0x7f3a341059a0 2024-09-19T06:09:34.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.241+0000 7f3a3a394640 1 --2- 172.21.15.53:0/3562707432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a341059a0 0x7f3a34199960 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3a30030380 tx=0x7f3a30032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:34.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.242+0000 7f3a277fe640 1 -- 172.21.15.53:0/3562707432 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f3a30043070 con 0x7f3a341059a0 2024-09-19T06:09:34.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.242+0000 7f3a277fe640 1 -- 172.21.15.53:0/3562707432 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3a3003e480 con 0x7f3a341059a0 2024-09-19T06:09:34.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.242+0000 7f3a3c5f8640 1 -- 172.21.15.53:0/3562707432 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a3419a130 con 0x7f3a341059a0 2024-09-19T06:09:34.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.242+0000 7f3a277fe640 1 -- 172.21.15.53:0/3562707432 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f3a3003c510 con 0x7f3a341059a0 2024-09-19T06:09:34.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.242+0000 7f3a3c5f8640 1 -- 172.21.15.53:0/3562707432 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a3419ce20 con 0x7f3a341059a0 2024-09-19T06:09:34.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.242+0000 7f3a277fe640 1 -- 172.21.15.53:0/3562707432 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3a30046070 con 0x7f3a341059a0 2024-09-19T06:09:34.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.242+0000 7f3a277fe640 1 -- 172.21.15.53:0/3562707432 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3a300489f0 con 0x7f3a341059a0 2024-09-19T06:09:34.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.243+0000 7f3a3c5f8640 1 -- 172.21.15.53:0/3562707432 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a3410a0d0 con 0x7f3a341059a0 2024-09-19T06:09:34.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.245+0000 7f3a277fe640 1 -- 172.21.15.53:0/3562707432 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f3a3003e630 con 0x7f3a341059a0 2024-09-19T06:09:34.892 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.307+0000 7f3a3c5f8640 1 -- 172.21.15.53:0/3562707432 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f3a3419d3b0 con 0x7f3a341059a0 2024-09-19T06:09:34.892 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.307+0000 7f3a277fe640 1 -- 172.21.15.53:0/3562707432 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1299 (secure 0 0 0) 0x7f3a3003e810 con 0x7f3a341059a0 2024-09-19T06:09:34.892 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.308+0000 7f3a3c5f8640 1 -- 172.21.15.53:0/3562707432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a341059a0 msgr2=0x7f3a34199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:34.892 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.308+0000 7f3a3c5f8640 1 --2- 172.21.15.53:0/3562707432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a341059a0 0x7f3a34199960 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3a30030380 tx=0x7f3a30032670 comp rx=0 tx=0).stop 2024-09-19T06:09:34.892 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.308+0000 7f3a3c5f8640 1 -- 172.21.15.53:0/3562707432 shutdown_connections 2024-09-19T06:09:34.892 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.308+0000 7f3a3c5f8640 1 --2- 172.21.15.53:0/3562707432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a341059a0 0x7f3a34199960 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.892 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.308+0000 7f3a3c5f8640 1 -- 172.21.15.53:0/3562707432 >> 172.21.15.53:0/3562707432 conn(0x7f3a34101190 msgr2=0x7f3a34101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:34.892 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.308+0000 7f3a3c5f8640 1 -- 172.21.15.53:0/3562707432 shutdown_connections 2024-09-19T06:09:34.892 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:34.309+0000 7f3a3c5f8640 1 -- 172.21.15.53:0/3562707432 wait complete. 2024-09-19T06:09:34.892 INFO:teuthology.orchestra.run.smithi053.stdout:mgr not available, waiting (1/15)... 2024-09-19T06:09:37.579 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:37 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/3299558862' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T06:09:37.947 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2024-09-19T06:09:37.947 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout { 2024-09-19T06:09:37.947 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "fsid": "9f2876d0-764d-11ef-bceb-c7b262605968", 2024-09-19T06:09:37.947 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T06:09:37.947 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T06:09:37.947 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T06:09:37.947 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T06:09:37.947 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:09:37.947 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T06:09:37.948 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T06:09:37.948 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 0 2024-09-19T06:09:37.948 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2024-09-19T06:09:37.948 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T06:09:37.948 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "smithi053" 2024-09-19T06:09:37.948 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2024-09-19T06:09:37.948 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-19T06:09:37.948 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T06:09:37.948 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:09:37.948 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T06:09:37.948 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T06:09:37.948 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:09:37.948 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T06:09:37.948 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:09:37.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T06:09:37.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T06:09:37.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T06:09:37.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T06:09:37.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T06:09:37.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T06:09:37.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:09:37.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T06:09:37.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T06:09:37.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T06:09:37.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T06:09:37.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T06:09:37.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T06:09:37.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T06:09:37.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T06:09:37.950 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T06:09:37.950 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:09:37.950 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T06:09:37.950 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:09:37.950 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T06:09:26:797704+0000", 2024-09-19T06:09:37.950 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T06:09:37.950 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T06:09:37.950 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:09:37.950 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T06:09:37.950 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T06:09:37.950 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T06:09:37.950 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T06:09:37.950 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T06:09:37.950 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T06:09:37.951 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T06:09:37.951 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2024-09-19T06:09:37.951 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T06:09:37.951 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:09:37.951 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T06:09:37.951 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:09:37.951 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T06:09:26.798327+0000", 2024-09-19T06:09:37.951 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T06:09:37.951 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:09:37.951 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T06:09:37.951 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout } 2024-09-19T06:09:37.951 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.292+0000 7f52be2e9640 1 Processor -- start 2024-09-19T06:09:37.951 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.292+0000 7f52be2e9640 1 -- start start 2024-09-19T06:09:37.951 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.292+0000 7f52be2e9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b81059d0 0x7f52b8105dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:37.951 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.292+0000 7f52be2e9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81063a0 con 0x7f52b81059d0 2024-09-19T06:09:37.951 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.292+0000 7f52b7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b81059d0 0x7f52b8105dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:37.952 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.292+0000 7f52b7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b81059d0 0x7f52b8105dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39828/0 (socket says 172.21.15.53:39828) 2024-09-19T06:09:37.952 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.292+0000 7f52b7fff640 1 -- 172.21.15.53:0/1857636718 learned_addr learned my addr 172.21.15.53:0/1857636718 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:09:37.952 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.293+0000 7f52b7fff640 1 -- 172.21.15.53:0/1857636718 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b8106bd0 con 0x7f52b81059d0 2024-09-19T06:09:37.952 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.293+0000 7f52b7fff640 1 --2- 172.21.15.53:0/1857636718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b81059d0 0x7f52b8105dd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f52a800bac0 tx=0x7f52a802c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b586e80a840bc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:37.952 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.293+0000 7f52b6ffd640 1 -- 172.21.15.53:0/1857636718 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f52a80325a0 con 0x7f52b81059d0 2024-09-19T06:09:37.952 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.294+0000 7f52b6ffd640 1 -- 172.21.15.53:0/1857636718 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f52a8038070 con 0x7f52b81059d0 2024-09-19T06:09:37.952 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.294+0000 7f52b6ffd640 1 -- 172.21.15.53:0/1857636718 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f52a8033bc0 con 0x7f52b81059d0 2024-09-19T06:09:37.952 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.294+0000 7f52be2e9640 1 -- 172.21.15.53:0/1857636718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b81059d0 msgr2=0x7f52b8105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:37.952 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.294+0000 7f52be2e9640 1 --2- 172.21.15.53:0/1857636718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b81059d0 0x7f52b8105dd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f52a800bac0 tx=0x7f52a802c920 comp rx=0 tx=0).stop 2024-09-19T06:09:37.952 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.294+0000 7f52be2e9640 1 -- 172.21.15.53:0/1857636718 shutdown_connections 2024-09-19T06:09:37.952 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.294+0000 7f52be2e9640 1 --2- 172.21.15.53:0/1857636718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b81059d0 0x7f52b8105dd0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:37.952 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.294+0000 7f52be2e9640 1 -- 172.21.15.53:0/1857636718 >> 172.21.15.53:0/1857636718 conn(0x7f52b81011a0 msgr2=0x7f52b81035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:37.952 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.294+0000 7f52be2e9640 1 -- 172.21.15.53:0/1857636718 shutdown_connections 2024-09-19T06:09:37.953 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.294+0000 7f52be2e9640 1 -- 172.21.15.53:0/1857636718 wait complete. 2024-09-19T06:09:37.953 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.295+0000 7f52be2e9640 1 Processor -- start 2024-09-19T06:09:37.953 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.295+0000 7f52be2e9640 1 -- start start 2024-09-19T06:09:37.953 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.295+0000 7f52be2e9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b81059d0 0x7f52b819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:37.953 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.295+0000 7f52be2e9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f52b8079850 con 0x7f52b81059d0 2024-09-19T06:09:37.953 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.295+0000 7f52b7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b81059d0 0x7f52b819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:37.953 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.295+0000 7f52b7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b81059d0 0x7f52b819ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39836/0 (socket says 172.21.15.53:39836) 2024-09-19T06:09:37.953 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.296+0000 7f52b7fff640 1 -- 172.21.15.53:0/3299558862 learned_addr learned my addr 172.21.15.53:0/3299558862 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:09:37.953 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.296+0000 7f52b7fff640 1 -- 172.21.15.53:0/3299558862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b819d1f0 con 0x7f52b81059d0 2024-09-19T06:09:37.953 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.296+0000 7f52b7fff640 1 --2- 172.21.15.53:0/3299558862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b81059d0 0x7f52b819ccb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f52a800b2c0 tx=0x7f52a8032ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:37.953 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.296+0000 7f52b57fa640 1 -- 172.21.15.53:0/3299558862 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f52a8042070 con 0x7f52b81059d0 2024-09-19T06:09:37.953 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.296+0000 7f52b57fa640 1 -- 172.21.15.53:0/3299558862 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f52a803ea20 con 0x7f52b81059d0 2024-09-19T06:09:37.953 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.296+0000 7f52b57fa640 1 -- 172.21.15.53:0/3299558862 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f52a803cb20 con 0x7f52b81059d0 2024-09-19T06:09:37.953 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.296+0000 7f52be2e9640 1 -- 172.21.15.53:0/3299558862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b819d480 con 0x7f52b81059d0 2024-09-19T06:09:37.954 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.296+0000 7f52be2e9640 1 -- 172.21.15.53:0/3299558862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b81a0170 con 0x7f52b81059d0 2024-09-19T06:09:37.954 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.297+0000 7f52b57fa640 1 -- 172.21.15.53:0/3299558862 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f52a8045070 con 0x7f52b81059d0 2024-09-19T06:09:37.954 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.297+0000 7f52b57fa640 1 -- 172.21.15.53:0/3299558862 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f52a8030050 con 0x7f52b81059d0 2024-09-19T06:09:37.954 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.297+0000 7f52be2e9640 1 -- 172.21.15.53:0/3299558862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52b8105e50 con 0x7f52b81059d0 2024-09-19T06:09:37.954 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.300+0000 7f52b57fa640 1 -- 172.21.15.53:0/3299558862 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f52a803c410 con 0x7f52b81059d0 2024-09-19T06:09:37.954 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.362+0000 7f52be2e9640 1 -- 172.21.15.53:0/3299558862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f52b8191d60 con 0x7f52b81059d0 2024-09-19T06:09:37.954 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.363+0000 7f52b57fa640 1 -- 172.21.15.53:0/3299558862 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1299 (secure 0 0 0) 0x7f52a803ebd0 con 0x7f52b81059d0 2024-09-19T06:09:37.954 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.364+0000 7f52be2e9640 1 -- 172.21.15.53:0/3299558862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b81059d0 msgr2=0x7f52b819ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:37.954 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.364+0000 7f52be2e9640 1 --2- 172.21.15.53:0/3299558862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b81059d0 0x7f52b819ccb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f52a800b2c0 tx=0x7f52a8032ad0 comp rx=0 tx=0).stop 2024-09-19T06:09:37.954 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.364+0000 7f52be2e9640 1 -- 172.21.15.53:0/3299558862 shutdown_connections 2024-09-19T06:09:37.954 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.364+0000 7f52be2e9640 1 --2- 172.21.15.53:0/3299558862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b81059d0 0x7f52b819ccb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:37.954 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.364+0000 7f52be2e9640 1 -- 172.21.15.53:0/3299558862 >> 172.21.15.53:0/3299558862 conn(0x7f52b81011a0 msgr2=0x7f52b818eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:37.954 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.364+0000 7f52be2e9640 1 -- 172.21.15.53:0/3299558862 shutdown_connections 2024-09-19T06:09:37.954 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:37.364+0000 7f52be2e9640 1 -- 172.21.15.53:0/3299558862 wait complete. 2024-09-19T06:09:37.954 INFO:teuthology.orchestra.run.smithi053.stdout:mgr not available, waiting (2/15)... 2024-09-19T06:09:39.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:38 smithi053 ceph-mon[27968]: Activating manager daemon smithi053.trakvg 2024-09-19T06:09:39.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:38 smithi053 ceph-mon[27968]: mgrmap e2: smithi053.trakvg(active, starting, since 0.00320861s) 2024-09-19T06:09:39.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:38 smithi053 ceph-mon[27968]: from='mgr.14100 172.21.15.53:0/3640368587' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T06:09:39.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:38 smithi053 ceph-mon[27968]: from='mgr.14100 172.21.15.53:0/3640368587' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T06:09:39.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:38 smithi053 ceph-mon[27968]: from='mgr.14100 172.21.15.53:0/3640368587' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T06:09:39.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:38 smithi053 ceph-mon[27968]: from='mgr.14100 172.21.15.53:0/3640368587' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata", "id": "smithi053"}]: dispatch 2024-09-19T06:09:39.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:38 smithi053 ceph-mon[27968]: from='mgr.14100 172.21.15.53:0/3640368587' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mgr metadata", "who": "smithi053.trakvg", "id": "smithi053.trakvg"}]: dispatch 2024-09-19T06:09:39.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:38 smithi053 ceph-mon[27968]: Manager daemon smithi053.trakvg is now available 2024-09-19T06:09:39.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:38 smithi053 ceph-mon[27968]: from='mgr.14100 172.21.15.53:0/3640368587' entity='mgr.smithi053.trakvg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi053.trakvg/mirror_snapshot_schedule"}]: dispatch 2024-09-19T06:09:39.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:38 smithi053 ceph-mon[27968]: from='mgr.14100 172.21.15.53:0/3640368587' entity='mgr.smithi053.trakvg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi053.trakvg/trash_purge_schedule"}]: dispatch 2024-09-19T06:09:39.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:38 smithi053 ceph-mon[27968]: from='mgr.14100 172.21.15.53:0/3640368587' entity='mgr.smithi053.trakvg' 2024-09-19T06:09:39.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:38 smithi053 ceph-mon[27968]: from='mgr.14100 172.21.15.53:0/3640368587' entity='mgr.smithi053.trakvg' 2024-09-19T06:09:39.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:38 smithi053 ceph-mon[27968]: from='mgr.14100 172.21.15.53:0/3640368587' entity='mgr.smithi053.trakvg' 2024-09-19T06:09:41.056 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:40 smithi053 ceph-mon[27968]: mgrmap e3: smithi053.trakvg(active, since 1.00689s) 2024-09-19T06:09:42.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:41 smithi053 ceph-mon[27968]: mgrmap e4: smithi053.trakvg(active, since 2s) 2024-09-19T06:09:42.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:41 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/942958363' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T06:09:47.995 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2024-09-19T06:09:47.995 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout { 2024-09-19T06:09:47.995 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "fsid": "9f2876d0-764d-11ef-bceb-c7b262605968", 2024-09-19T06:09:47.995 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T06:09:47.995 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T06:09:47.995 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T06:09:47.995 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T06:09:47.996 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:09:47.996 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T06:09:47.996 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T06:09:47.996 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 0 2024-09-19T06:09:47.996 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2024-09-19T06:09:47.996 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T06:09:47.996 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "smithi053" 2024-09-19T06:09:47.996 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2024-09-19T06:09:47.996 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-19T06:09:47.996 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T06:09:47.996 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:09:47.996 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T06:09:47.996 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T06:09:47.997 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:09:47.997 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T06:09:47.997 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:09:47.997 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T06:09:47.997 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T06:09:47.997 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T06:09:48.006 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T06:09:48.007 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T06:09:48.007 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T06:09:48.007 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:09:48.007 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T06:09:48.007 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T06:09:48.007 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T06:09:48.007 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T06:09:48.007 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T06:09:48.007 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T06:09:48.007 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T06:09:48.007 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T06:09:48.007 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T06:09:48.007 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:09:48.007 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T06:09:48.008 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:09:48.008 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T06:09:26:797704+0000", 2024-09-19T06:09:48.008 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T06:09:48.008 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T06:09:48.008 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:09:48.008 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T06:09:48.008 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T06:09:48.008 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T06:09:48.008 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T06:09:48.008 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T06:09:48.008 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T06:09:48.008 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T06:09:48.008 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2024-09-19T06:09:48.009 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T06:09:48.009 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:09:48.009 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T06:09:48.009 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:09:48.009 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T06:09:26.798327+0000", 2024-09-19T06:09:48.009 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T06:09:48.009 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:09:48.009 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T06:09:48.009 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout } 2024-09-19T06:09:48.009 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.143+0000 7fa050a27640 1 Processor -- start 2024-09-19T06:09:48.009 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.143+0000 7fa050a27640 1 -- start start 2024-09-19T06:09:48.009 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.144+0000 7fa050a27640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa04c108cc0 0x7fa04c1090c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:48.009 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.144+0000 7fa050a27640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c109690 con 0x7fa04c108cc0 2024-09-19T06:09:48.010 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.144+0000 7fa04a59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa04c108cc0 0x7fa04c1090c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:48.010 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.144+0000 7fa04a59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa04c108cc0 0x7fa04c1090c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39918/0 (socket says 172.21.15.53:39918) 2024-09-19T06:09:48.010 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.144+0000 7fa04a59c640 1 -- 172.21.15.53:0/4252541629 learned_addr learned my addr 172.21.15.53:0/4252541629 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:09:48.010 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.144+0000 7fa04a59c640 1 -- 172.21.15.53:0/4252541629 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04c109e70 con 0x7fa04c108cc0 2024-09-19T06:09:48.010 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.145+0000 7fa04a59c640 1 --2- 172.21.15.53:0/4252541629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa04c108cc0 0x7fa04c1090c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa03c009820 tx=0x7fa03c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad86181ac71b4657 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:48.010 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.145+0000 7fa04959a640 1 -- 172.21.15.53:0/4252541629 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fa03c037070 con 0x7fa04c108cc0 2024-09-19T06:09:48.010 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.145+0000 7fa04959a640 1 -- 172.21.15.53:0/4252541629 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa03c02ad40 con 0x7fa04c108cc0 2024-09-19T06:09:48.010 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.145+0000 7fa04959a640 1 -- 172.21.15.53:0/4252541629 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fa03c030670 con 0x7fa04c108cc0 2024-09-19T06:09:48.010 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.146+0000 7fa050a27640 1 -- 172.21.15.53:0/4252541629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa04c108cc0 msgr2=0x7fa04c1090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:48.010 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.146+0000 7fa050a27640 1 --2- 172.21.15.53:0/4252541629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa04c108cc0 0x7fa04c1090c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa03c009820 tx=0x7fa03c02a270 comp rx=0 tx=0).stop 2024-09-19T06:09:48.010 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.146+0000 7fa050a27640 1 -- 172.21.15.53:0/4252541629 shutdown_connections 2024-09-19T06:09:48.011 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.146+0000 7fa050a27640 1 --2- 172.21.15.53:0/4252541629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa04c108cc0 0x7fa04c1090c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:48.011 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.146+0000 7fa050a27640 1 -- 172.21.15.53:0/4252541629 >> 172.21.15.53:0/4252541629 conn(0x7fa04c07c410 msgr2=0x7fa04c106d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:48.011 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.146+0000 7fa050a27640 1 -- 172.21.15.53:0/4252541629 shutdown_connections 2024-09-19T06:09:48.011 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.146+0000 7fa050a27640 1 -- 172.21.15.53:0/4252541629 wait complete. 2024-09-19T06:09:48.011 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.147+0000 7fa050a27640 1 Processor -- start 2024-09-19T06:09:48.011 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.147+0000 7fa050a27640 1 -- start start 2024-09-19T06:09:48.011 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.147+0000 7fa050a27640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa04c108cc0 0x7fa04c19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:48.011 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.147+0000 7fa050a27640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c10d220 con 0x7fa04c108cc0 2024-09-19T06:09:48.011 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.147+0000 7fa04a59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa04c108cc0 0x7fa04c19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:48.012 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.147+0000 7fa04a59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa04c108cc0 0x7fa04c19cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39930/0 (socket says 172.21.15.53:39930) 2024-09-19T06:09:48.012 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.147+0000 7fa04a59c640 1 -- 172.21.15.53:0/942958363 learned_addr learned my addr 172.21.15.53:0/942958363 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:09:48.012 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.148+0000 7fa04a59c640 1 -- 172.21.15.53:0/942958363 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04c19d120 con 0x7fa04c108cc0 2024-09-19T06:09:48.012 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.148+0000 7fa04a59c640 1 --2- 172.21.15.53:0/942958363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa04c108cc0 0x7fa04c19cbe0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa03c02a790 tx=0x7fa03c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:48.012 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.148+0000 7fa03b7fe640 1 -- 172.21.15.53:0/942958363 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fa03c043070 con 0x7fa04c108cc0 2024-09-19T06:09:48.012 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.148+0000 7fa03b7fe640 1 -- 172.21.15.53:0/942958363 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa03c03e480 con 0x7fa04c108cc0 2024-09-19T06:09:48.012 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.148+0000 7fa050a27640 1 -- 172.21.15.53:0/942958363 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa04c19d3b0 con 0x7fa04c108cc0 2024-09-19T06:09:48.012 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.148+0000 7fa03b7fe640 1 -- 172.21.15.53:0/942958363 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fa03c03c510 con 0x7fa04c108cc0 2024-09-19T06:09:48.012 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.148+0000 7fa050a27640 1 -- 172.21.15.53:0/942958363 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa04c1a00a0 con 0x7fa04c108cc0 2024-09-19T06:09:48.012 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.149+0000 7fa03b7fe640 1 -- 172.21.15.53:0/942958363 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 4) ==== 46465+0+0 (secure 0 0 0) 0x7fa03c046070 con 0x7fa04c108cc0 2024-09-19T06:09:48.012 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.149+0000 7fa050a27640 1 -- 172.21.15.53:0/942958363 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa04c10d3a0 con 0x7fa04c108cc0 2024-09-19T06:09:48.012 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.149+0000 7fa03b7fe640 1 --2- 172.21.15.53:0/942958363 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7fa0340391e0 0x7fa03403b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:48.012 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.149+0000 7fa03b7fe640 1 -- 172.21.15.53:0/942958363 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa03c06bdf0 con 0x7fa04c108cc0 2024-09-19T06:09:48.013 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.150+0000 7fa049d9b640 1 --2- 172.21.15.53:0/942958363 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7fa0340391e0 0x7fa03403b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:48.013 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.150+0000 7fa049d9b640 1 --2- 172.21.15.53:0/942958363 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7fa0340391e0 0x7fa03403b6a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa0400098c0 tx=0x7fa040006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:48.013 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.155+0000 7fa03b7fe640 1 -- 172.21.15.53:0/942958363 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa03c0391a0 con 0x7fa04c108cc0 2024-09-19T06:09:48.013 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.399+0000 7fa050a27640 1 -- 172.21.15.53:0/942958363 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa04c1a0730 con 0x7fa04c108cc0 2024-09-19T06:09:48.013 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.400+0000 7fa03b7fe640 1 -- 172.21.15.53:0/942958363 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1298 (secure 0 0 0) 0x7fa03c03e630 con 0x7fa04c108cc0 2024-09-19T06:09:48.013 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.402+0000 7fa050a27640 1 -- 172.21.15.53:0/942958363 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7fa0340391e0 msgr2=0x7fa03403b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:48.013 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.402+0000 7fa050a27640 1 --2- 172.21.15.53:0/942958363 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7fa0340391e0 0x7fa03403b6a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa0400098c0 tx=0x7fa040006db0 comp rx=0 tx=0).stop 2024-09-19T06:09:48.013 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.402+0000 7fa050a27640 1 -- 172.21.15.53:0/942958363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa04c108cc0 msgr2=0x7fa04c19cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:48.013 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.402+0000 7fa050a27640 1 --2- 172.21.15.53:0/942958363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa04c108cc0 0x7fa04c19cbe0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa03c02a790 tx=0x7fa03c032670 comp rx=0 tx=0).stop 2024-09-19T06:09:48.013 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.402+0000 7fa050a27640 1 -- 172.21.15.53:0/942958363 shutdown_connections 2024-09-19T06:09:48.013 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.402+0000 7fa050a27640 1 --2- 172.21.15.53:0/942958363 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7fa0340391e0 0x7fa03403b6a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:48.013 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.402+0000 7fa050a27640 1 --2- 172.21.15.53:0/942958363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa04c108cc0 0x7fa04c19cbe0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:48.013 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.402+0000 7fa050a27640 1 -- 172.21.15.53:0/942958363 >> 172.21.15.53:0/942958363 conn(0x7fa04c07c410 msgr2=0x7fa04c106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:48.013 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.402+0000 7fa050a27640 1 -- 172.21.15.53:0/942958363 shutdown_connections 2024-09-19T06:09:48.014 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:41.402+0000 7fa050a27640 1 -- 172.21.15.53:0/942958363 wait complete. 2024-09-19T06:09:48.014 INFO:teuthology.orchestra.run.smithi053.stdout:mgr is available 2024-09-19T06:09:51.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:51 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/2983606951' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T06:09:52.509 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2024-09-19T06:09:52.509 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T06:09:52.509 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout fsid = 9f2876d0-764d-11ef-bceb-c7b262605968 2024-09-19T06:09:52.509 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.53:3300,v1:172.21.15.53:6789] 2024-09-19T06:09:52.509 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T06:09:52.509 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T06:09:52.509 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T06:09:52.509 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T06:09:52.509 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2024-09-19T06:09:52.509 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T06:09:52.510 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T06:09:52.510 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2024-09-19T06:09:52.510 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T06:09:52.510 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T06:09:52.510 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T06:09:52.510 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.916+0000 7fd938b59640 1 Processor -- start 2024-09-19T06:09:52.510 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.916+0000 7fd938b59640 1 -- start start 2024-09-19T06:09:52.510 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.916+0000 7fd938b59640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd934108cc0 0x7fd9341090c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:52.510 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.916+0000 7fd938b59640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd934109690 con 0x7fd934108cc0 2024-09-19T06:09:52.510 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.916+0000 7fd93259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd934108cc0 0x7fd9341090c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:52.510 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.917+0000 7fd93259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd934108cc0 0x7fd9341090c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41618/0 (socket says 172.21.15.53:41618) 2024-09-19T06:09:52.510 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.917+0000 7fd93259c640 1 -- 172.21.15.53:0/2666792502 learned_addr learned my addr 172.21.15.53:0/2666792502 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:09:52.511 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.917+0000 7fd93259c640 1 -- 172.21.15.53:0/2666792502 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd934109e70 con 0x7fd934108cc0 2024-09-19T06:09:52.511 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.917+0000 7fd93259c640 1 --2- 172.21.15.53:0/2666792502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd934108cc0 0x7fd9341090c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd924009820 tx=0x7fd92402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec82becf4572d992 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:52.511 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.918+0000 7fd931d9b640 1 -- 172.21.15.53:0/2666792502 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fd924037070 con 0x7fd934108cc0 2024-09-19T06:09:52.511 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.918+0000 7fd931d9b640 1 -- 172.21.15.53:0/2666792502 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fd92402ad40 con 0x7fd934108cc0 2024-09-19T06:09:52.511 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.918+0000 7fd931d9b640 1 -- 172.21.15.53:0/2666792502 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fd924030670 con 0x7fd934108cc0 2024-09-19T06:09:52.511 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.918+0000 7fd938b59640 1 -- 172.21.15.53:0/2666792502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd934108cc0 msgr2=0x7fd9341090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:52.511 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.918+0000 7fd938b59640 1 --2- 172.21.15.53:0/2666792502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd934108cc0 0x7fd9341090c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd924009820 tx=0x7fd92402a270 comp rx=0 tx=0).stop 2024-09-19T06:09:52.511 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.918+0000 7fd938b59640 1 -- 172.21.15.53:0/2666792502 shutdown_connections 2024-09-19T06:09:52.511 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.918+0000 7fd938b59640 1 --2- 172.21.15.53:0/2666792502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd934108cc0 0x7fd9341090c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.511 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.918+0000 7fd938b59640 1 -- 172.21.15.53:0/2666792502 >> 172.21.15.53:0/2666792502 conn(0x7fd93407c410 msgr2=0x7fd934106d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:52.511 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.918+0000 7fd938b59640 1 -- 172.21.15.53:0/2666792502 shutdown_connections 2024-09-19T06:09:52.511 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.918+0000 7fd938b59640 1 -- 172.21.15.53:0/2666792502 wait complete. 2024-09-19T06:09:52.511 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.919+0000 7fd938b59640 1 Processor -- start 2024-09-19T06:09:52.512 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.919+0000 7fd938b59640 1 -- start start 2024-09-19T06:09:52.512 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.920+0000 7fd938b59640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd934108cc0 0x7fd93419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:52.512 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.920+0000 7fd938b59640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd93410d220 con 0x7fd934108cc0 2024-09-19T06:09:52.512 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.920+0000 7fd93259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd934108cc0 0x7fd93419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:52.512 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.920+0000 7fd93259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd934108cc0 0x7fd93419cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41624/0 (socket says 172.21.15.53:41624) 2024-09-19T06:09:52.512 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.920+0000 7fd93259c640 1 -- 172.21.15.53:0/2983606951 learned_addr learned my addr 172.21.15.53:0/2983606951 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:09:52.512 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.920+0000 7fd93259c640 1 -- 172.21.15.53:0/2983606951 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd93419d120 con 0x7fd934108cc0 2024-09-19T06:09:52.512 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.920+0000 7fd93259c640 1 --2- 172.21.15.53:0/2983606951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd934108cc0 0x7fd93419cbe0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd924030380 tx=0x7fd924032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:52.512 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.920+0000 7fd92bfff640 1 -- 172.21.15.53:0/2983606951 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fd924043070 con 0x7fd934108cc0 2024-09-19T06:09:52.512 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.920+0000 7fd92bfff640 1 -- 172.21.15.53:0/2983606951 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fd92403e480 con 0x7fd934108cc0 2024-09-19T06:09:52.512 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.920+0000 7fd938b59640 1 -- 172.21.15.53:0/2983606951 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd93419d3b0 con 0x7fd934108cc0 2024-09-19T06:09:52.512 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.920+0000 7fd92bfff640 1 -- 172.21.15.53:0/2983606951 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fd92403c510 con 0x7fd934108cc0 2024-09-19T06:09:52.512 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.921+0000 7fd938b59640 1 -- 172.21.15.53:0/2983606951 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9341a00a0 con 0x7fd934108cc0 2024-09-19T06:09:52.513 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.921+0000 7fd92bfff640 1 -- 172.21.15.53:0/2983606951 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 4) ==== 46465+0+0 (secure 0 0 0) 0x7fd924046070 con 0x7fd934108cc0 2024-09-19T06:09:52.513 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.922+0000 7fd92bfff640 1 --2- 172.21.15.53:0/2983606951 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7fd920039190 0x7fd92003b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:52.513 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.922+0000 7fd92bfff640 1 -- 172.21.15.53:0/2983606951 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd92406be80 con 0x7fd934108cc0 2024-09-19T06:09:52.513 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.922+0000 7fd938b59640 1 -- 172.21.15.53:0/2983606951 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd93410d3a0 con 0x7fd934108cc0 2024-09-19T06:09:52.513 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.922+0000 7fd929d9b640 1 --2- 172.21.15.53:0/2983606951 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7fd920039190 0x7fd92003b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:52.513 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.922+0000 7fd929d9b640 1 --2- 172.21.15.53:0/2983606951 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7fd920039190 0x7fd92003b650 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd91c0098c0 tx=0x7fd91c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:52.513 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:50.926+0000 7fd92bfff640 1 -- 172.21.15.53:0/2983606951 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd92406b8a0 con 0x7fd934108cc0 2024-09-19T06:09:52.513 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:51.107+0000 7fd938b59640 1 -- 172.21.15.53:0/2983606951 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fd9341a0270 con 0x7fd934108cc0 2024-09-19T06:09:52.513 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:51.108+0000 7fd92bfff640 1 -- 172.21.15.53:0/2983606951 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+338 (secure 0 0 0) 0x7fd92403c7c0 con 0x7fd934108cc0 2024-09-19T06:09:52.513 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:51.110+0000 7fd938b59640 1 -- 172.21.15.53:0/2983606951 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7fd920039190 msgr2=0x7fd92003b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:52.513 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:51.110+0000 7fd938b59640 1 --2- 172.21.15.53:0/2983606951 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7fd920039190 0x7fd92003b650 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd91c0098c0 tx=0x7fd91c006db0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.513 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:51.110+0000 7fd938b59640 1 -- 172.21.15.53:0/2983606951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd934108cc0 msgr2=0x7fd93419cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:52.513 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:51.110+0000 7fd938b59640 1 --2- 172.21.15.53:0/2983606951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd934108cc0 0x7fd93419cbe0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd924030380 tx=0x7fd924032670 comp rx=0 tx=0).stop 2024-09-19T06:09:52.514 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:51.110+0000 7fd938b59640 1 -- 172.21.15.53:0/2983606951 shutdown_connections 2024-09-19T06:09:52.514 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:51.110+0000 7fd938b59640 1 --2- 172.21.15.53:0/2983606951 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7fd920039190 0x7fd92003b650 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.514 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:51.110+0000 7fd938b59640 1 --2- 172.21.15.53:0/2983606951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd934108cc0 0x7fd93419cbe0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.514 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:51.110+0000 7fd938b59640 1 -- 172.21.15.53:0/2983606951 >> 172.21.15.53:0/2983606951 conn(0x7fd93407c410 msgr2=0x7fd934106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:52.514 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:51.110+0000 7fd938b59640 1 -- 172.21.15.53:0/2983606951 shutdown_connections 2024-09-19T06:09:52.514 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:51.110+0000 7fd938b59640 1 -- 172.21.15.53:0/2983606951 wait complete. 2024-09-19T06:09:52.514 INFO:teuthology.orchestra.run.smithi053.stdout:Enabling cephadm module... 2024-09-19T06:09:53.500 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:53 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/2330471129' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-19T06:09:53.782 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.897+0000 7f779740a640 1 Processor -- start 2024-09-19T06:09:53.782 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.897+0000 7f779740a640 1 -- start start 2024-09-19T06:09:53.782 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.898+0000 7f779740a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7790106ac0 0x7f7790106ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:53.782 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.898+0000 7f779740a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7790107400 con 0x7f7790106ac0 2024-09-19T06:09:53.782 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.898+0000 7f77951a6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7790106ac0 0x7f7790106ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:53.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.898+0000 7f77951a6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7790106ac0 0x7f7790106ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41700/0 (socket says 172.21.15.53:41700) 2024-09-19T06:09:53.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.898+0000 7f77951a6640 1 -- 172.21.15.53:0/2078512793 learned_addr learned my addr 172.21.15.53:0/2078512793 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:09:53.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.898+0000 7f77951a6640 1 -- 172.21.15.53:0/2078512793 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7790107580 con 0x7f7790106ac0 2024-09-19T06:09:53.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.899+0000 7f77951a6640 1 --2- 172.21.15.53:0/2078512793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7790106ac0 0x7f7790106ec0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7780009e80 tx=0x7f778002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4dcbb1ddbb52ea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:53.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.899+0000 7f778ffff640 1 -- 172.21.15.53:0/2078512793 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f7780036070 con 0x7f7790106ac0 2024-09-19T06:09:53.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.899+0000 7f778ffff640 1 -- 172.21.15.53:0/2078512793 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7780031890 con 0x7f7790106ac0 2024-09-19T06:09:53.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.899+0000 7f778ffff640 1 -- 172.21.15.53:0/2078512793 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f77800078c0 con 0x7f7790106ac0 2024-09-19T06:09:53.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.900+0000 7f779740a640 1 -- 172.21.15.53:0/2078512793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7790106ac0 msgr2=0x7f7790106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:53.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.900+0000 7f779740a640 1 --2- 172.21.15.53:0/2078512793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7790106ac0 0x7f7790106ec0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7780009e80 tx=0x7f778002a7a0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.900+0000 7f779740a640 1 -- 172.21.15.53:0/2078512793 shutdown_connections 2024-09-19T06:09:53.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.900+0000 7f779740a640 1 --2- 172.21.15.53:0/2078512793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7790106ac0 0x7f7790106ec0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.900+0000 7f779740a640 1 -- 172.21.15.53:0/2078512793 >> 172.21.15.53:0/2078512793 conn(0x7f7790102330 msgr2=0x7f7790104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:53.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.900+0000 7f779740a640 1 -- 172.21.15.53:0/2078512793 shutdown_connections 2024-09-19T06:09:53.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.900+0000 7f779740a640 1 -- 172.21.15.53:0/2078512793 wait complete. 2024-09-19T06:09:53.784 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.900+0000 7f779740a640 1 Processor -- start 2024-09-19T06:09:53.784 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.901+0000 7f779740a640 1 -- start start 2024-09-19T06:09:53.784 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.901+0000 7f779740a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7790106ac0 0x7f779007c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:53.784 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.901+0000 7f779740a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f779010af30 con 0x7f7790106ac0 2024-09-19T06:09:53.784 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.901+0000 7f77951a6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7790106ac0 0x7f779007c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:53.784 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.901+0000 7f77951a6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7790106ac0 0x7f779007c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41702/0 (socket says 172.21.15.53:41702) 2024-09-19T06:09:53.784 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.901+0000 7f77951a6640 1 -- 172.21.15.53:0/2330471129 learned_addr learned my addr 172.21.15.53:0/2330471129 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:09:53.784 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.901+0000 7f77951a6640 1 -- 172.21.15.53:0/2330471129 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f779007c930 con 0x7f7790106ac0 2024-09-19T06:09:53.784 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.901+0000 7f77951a6640 1 --2- 172.21.15.53:0/2330471129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7790106ac0 0x7f779007c3f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7780031630 tx=0x7f7780031660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:53.784 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.902+0000 7f778e7fc640 1 -- 172.21.15.53:0/2330471129 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f7780045070 con 0x7f7790106ac0 2024-09-19T06:09:53.784 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.902+0000 7f778e7fc640 1 -- 172.21.15.53:0/2330471129 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f778003a070 con 0x7f7790106ac0 2024-09-19T06:09:53.784 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.902+0000 7f779740a640 1 -- 172.21.15.53:0/2330471129 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f779007ab20 con 0x7f7790106ac0 2024-09-19T06:09:53.785 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.902+0000 7f778e7fc640 1 -- 172.21.15.53:0/2330471129 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f778003f3d0 con 0x7f7790106ac0 2024-09-19T06:09:53.785 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.902+0000 7f779740a640 1 -- 172.21.15.53:0/2330471129 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f779007b060 con 0x7f7790106ac0 2024-09-19T06:09:53.785 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.902+0000 7f778e7fc640 1 -- 172.21.15.53:0/2330471129 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 4) ==== 46465+0+0 (secure 0 0 0) 0x7f7780048070 con 0x7f7790106ac0 2024-09-19T06:09:53.785 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.903+0000 7f779740a640 1 -- 172.21.15.53:0/2330471129 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7790106f40 con 0x7f7790106ac0 2024-09-19T06:09:53.785 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.903+0000 7f778e7fc640 1 --2- 172.21.15.53:0/2330471129 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7f7778039190 0x7f777803b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:53.785 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.903+0000 7f778e7fc640 1 -- 172.21.15.53:0/2330471129 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f778006c700 con 0x7f7790106ac0 2024-09-19T06:09:53.785 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.903+0000 7f77949a5640 1 --2- 172.21.15.53:0/2330471129 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7f7778039190 0x7f777803b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:53.785 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.903+0000 7f77949a5640 1 --2- 172.21.15.53:0/2330471129 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7f7778039190 0x7f777803b650 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f77840098c0 tx=0x7f7784006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:53.785 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:52.908+0000 7f778e7fc640 1 -- 172.21.15.53:0/2330471129 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7780031c50 con 0x7f7790106ac0 2024-09-19T06:09:53.785 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:53.133+0000 7f779740a640 1 -- 172.21.15.53:0/2330471129 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f779007b2e0 con 0x7f7790106ac0 2024-09-19T06:09:53.785 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:53.184+0000 7f778e7fc640 1 -- 172.21.15.53:0/2330471129 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f778004f020 con 0x7f7790106ac0 2024-09-19T06:09:53.785 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:53.186+0000 7f778e7fc640 1 -- 172.21.15.53:0/2330471129 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mgrmap(e 5) ==== 46476+0+0 (secure 0 0 0) 0x7f7780048c60 con 0x7f7790106ac0 2024-09-19T06:09:53.785 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:53.187+0000 7f779740a640 1 -- 172.21.15.53:0/2330471129 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7f7778039190 msgr2=0x7f777803b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:53.786 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:53.187+0000 7f779740a640 1 --2- 172.21.15.53:0/2330471129 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7f7778039190 0x7f777803b650 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f77840098c0 tx=0x7f7784006db0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.786 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:53.187+0000 7f779740a640 1 -- 172.21.15.53:0/2330471129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7790106ac0 msgr2=0x7f779007c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:53.786 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:53.187+0000 7f779740a640 1 --2- 172.21.15.53:0/2330471129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7790106ac0 0x7f779007c3f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7780031630 tx=0x7f7780031660 comp rx=0 tx=0).stop 2024-09-19T06:09:53.786 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:53.187+0000 7f779740a640 1 -- 172.21.15.53:0/2330471129 shutdown_connections 2024-09-19T06:09:53.786 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:53.187+0000 7f779740a640 1 --2- 172.21.15.53:0/2330471129 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7f7778039190 0x7f777803b650 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.786 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:53.187+0000 7f779740a640 1 --2- 172.21.15.53:0/2330471129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7790106ac0 0x7f779007c3f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.786 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:53.187+0000 7f779740a640 1 -- 172.21.15.53:0/2330471129 >> 172.21.15.53:0/2330471129 conn(0x7f7790102330 msgr2=0x7f7790102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:53.786 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:53.187+0000 7f779740a640 1 -- 172.21.15.53:0/2330471129 shutdown_connections 2024-09-19T06:09:53.786 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:53.187+0000 7f779740a640 1 -- 172.21.15.53:0/2330471129 wait complete. 2024-09-19T06:09:54.319 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:54 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/2330471129' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-19T06:09:54.320 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:54 smithi053 ceph-mon[27968]: mgrmap e5: smithi053.trakvg(active, since 14s) 2024-09-19T06:09:55.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:55 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/813584692' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T06:09:55.833 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout { 2024-09-19T06:09:55.833 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-19T06:09:55.833 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T06:09:55.833 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "active_name": "smithi053.trakvg", 2024-09-19T06:09:55.833 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T06:09:55.833 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout } 2024-09-19T06:09:55.834 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.145+0000 7fd247bf0640 1 Processor -- start 2024-09-19T06:09:55.834 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.146+0000 7fd247bf0640 1 -- start start 2024-09-19T06:09:55.834 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.146+0000 7fd247bf0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2401059d0 0x7fd240105dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:55.834 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.146+0000 7fd247bf0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd2401063a0 con 0x7fd2401059d0 2024-09-19T06:09:55.834 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.146+0000 7fd24598c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2401059d0 0x7fd240105dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:55.834 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.146+0000 7fd24598c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2401059d0 0x7fd240105dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41732/0 (socket says 172.21.15.53:41732) 2024-09-19T06:09:55.834 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.146+0000 7fd24598c640 1 -- 172.21.15.53:0/3843959267 learned_addr learned my addr 172.21.15.53:0/3843959267 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:09:55.834 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.146+0000 7fd24598c640 1 -- 172.21.15.53:0/3843959267 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd240106bd0 con 0x7fd2401059d0 2024-09-19T06:09:55.834 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.147+0000 7fd24598c640 1 --2- 172.21.15.53:0/3843959267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2401059d0 0x7fd240105dd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd23c009820 tx=0x7fd23c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed4674f3a43fd2b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:55.834 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.147+0000 7fd24498a640 1 -- 172.21.15.53:0/3843959267 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fd23c037070 con 0x7fd2401059d0 2024-09-19T06:09:55.834 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.147+0000 7fd24498a640 1 -- 172.21.15.53:0/3843959267 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fd23c02ad40 con 0x7fd2401059d0 2024-09-19T06:09:55.834 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.147+0000 7fd24498a640 1 -- 172.21.15.53:0/3843959267 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fd23c030670 con 0x7fd2401059d0 2024-09-19T06:09:55.834 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.148+0000 7fd247bf0640 1 -- 172.21.15.53:0/3843959267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2401059d0 msgr2=0x7fd240105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:55.834 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.148+0000 7fd247bf0640 1 --2- 172.21.15.53:0/3843959267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2401059d0 0x7fd240105dd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd23c009820 tx=0x7fd23c02a270 comp rx=0 tx=0).stop 2024-09-19T06:09:55.835 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.148+0000 7fd247bf0640 1 -- 172.21.15.53:0/3843959267 shutdown_connections 2024-09-19T06:09:55.835 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.148+0000 7fd247bf0640 1 --2- 172.21.15.53:0/3843959267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2401059d0 0x7fd240105dd0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:55.835 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.148+0000 7fd247bf0640 1 -- 172.21.15.53:0/3843959267 >> 172.21.15.53:0/3843959267 conn(0x7fd2401011a0 msgr2=0x7fd2401035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:55.835 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.148+0000 7fd247bf0640 1 -- 172.21.15.53:0/3843959267 shutdown_connections 2024-09-19T06:09:55.835 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.148+0000 7fd247bf0640 1 -- 172.21.15.53:0/3843959267 wait complete. 2024-09-19T06:09:55.835 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.149+0000 7fd247bf0640 1 Processor -- start 2024-09-19T06:09:55.835 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.149+0000 7fd247bf0640 1 -- start start 2024-09-19T06:09:55.835 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.149+0000 7fd247bf0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2401059d0 0x7fd24019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:55.835 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.149+0000 7fd247bf0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd240109f80 con 0x7fd2401059d0 2024-09-19T06:09:55.835 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.149+0000 7fd24598c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2401059d0 0x7fd24019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:55.835 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.149+0000 7fd24598c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2401059d0 0x7fd24019aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41744/0 (socket says 172.21.15.53:41744) 2024-09-19T06:09:55.835 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.149+0000 7fd24598c640 1 -- 172.21.15.53:0/813584692 learned_addr learned my addr 172.21.15.53:0/813584692 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:09:55.835 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.149+0000 7fd24598c640 1 -- 172.21.15.53:0/813584692 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd24019aff0 con 0x7fd2401059d0 2024-09-19T06:09:55.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.150+0000 7fd24598c640 1 --2- 172.21.15.53:0/813584692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2401059d0 0x7fd24019aab0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd23c02a790 tx=0x7fd23c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:55.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.150+0000 7fd236ffd640 1 -- 172.21.15.53:0/813584692 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fd23c043070 con 0x7fd2401059d0 2024-09-19T06:09:55.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.150+0000 7fd236ffd640 1 -- 172.21.15.53:0/813584692 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fd23c03e480 con 0x7fd2401059d0 2024-09-19T06:09:55.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.150+0000 7fd247bf0640 1 -- 172.21.15.53:0/813584692 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd24019b280 con 0x7fd2401059d0 2024-09-19T06:09:55.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.150+0000 7fd236ffd640 1 -- 172.21.15.53:0/813584692 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fd23c03c510 con 0x7fd2401059d0 2024-09-19T06:09:55.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.150+0000 7fd247bf0640 1 -- 172.21.15.53:0/813584692 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd24019df70 con 0x7fd2401059d0 2024-09-19T06:09:55.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.151+0000 7fd247bf0640 1 -- 172.21.15.53:0/813584692 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd24010a100 con 0x7fd2401059d0 2024-09-19T06:09:55.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.151+0000 7fd236ffd640 1 -- 172.21.15.53:0/813584692 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 46476+0+0 (secure 0 0 0) 0x7fd23c046070 con 0x7fd2401059d0 2024-09-19T06:09:55.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.151+0000 7fd236ffd640 1 --2- 172.21.15.53:0/813584692 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7fd22c039230 0x7fd22c03b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:55.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.151+0000 7fd236ffd640 1 -- 172.21.15.53:0/813584692 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd23c06bfe0 con 0x7fd2401059d0 2024-09-19T06:09:55.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.151+0000 7fd24518b640 1 -- 172.21.15.53:0/813584692 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7fd22c039230 msgr2=0x7fd22c03b6f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/475342876 2024-09-19T06:09:55.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.151+0000 7fd24518b640 1 --2- 172.21.15.53:0/813584692 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7fd22c039230 0x7fd22c03b6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T06:09:55.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.156+0000 7fd236ffd640 1 -- 172.21.15.53:0/813584692 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd23c046360 con 0x7fd2401059d0 2024-09-19T06:09:55.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.352+0000 7fd24518b640 1 -- 172.21.15.53:0/813584692 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7fd22c039230 msgr2=0x7fd22c03b6f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/475342876 2024-09-19T06:09:55.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.352+0000 7fd24518b640 1 --2- 172.21.15.53:0/813584692 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7fd22c039230 0x7fd22c03b6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T06:09:55.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.378+0000 7fd247bf0640 1 -- 172.21.15.53:0/813584692 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fd24002da30 con 0x7fd2401059d0 2024-09-19T06:09:55.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.378+0000 7fd236ffd640 1 -- 172.21.15.53:0/813584692 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+103 (secure 0 0 0) 0x7fd23c043210 con 0x7fd2401059d0 2024-09-19T06:09:55.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.380+0000 7fd247bf0640 1 -- 172.21.15.53:0/813584692 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7fd22c039230 msgr2=0x7fd22c03b6f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T06:09:55.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.380+0000 7fd247bf0640 1 --2- 172.21.15.53:0/813584692 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7fd22c039230 0x7fd22c03b6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:55.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.380+0000 7fd247bf0640 1 -- 172.21.15.53:0/813584692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2401059d0 msgr2=0x7fd24019aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:55.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.380+0000 7fd247bf0640 1 --2- 172.21.15.53:0/813584692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2401059d0 0x7fd24019aab0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd23c02a790 tx=0x7fd23c032670 comp rx=0 tx=0).stop 2024-09-19T06:09:55.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.380+0000 7fd247bf0640 1 -- 172.21.15.53:0/813584692 shutdown_connections 2024-09-19T06:09:55.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.380+0000 7fd247bf0640 1 --2- 172.21.15.53:0/813584692 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7fd22c039230 0x7fd22c03b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:55.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.380+0000 7fd247bf0640 1 --2- 172.21.15.53:0/813584692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2401059d0 0x7fd24019aab0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:55.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.380+0000 7fd247bf0640 1 -- 172.21.15.53:0/813584692 >> 172.21.15.53:0/813584692 conn(0x7fd2401011a0 msgr2=0x7fd240102070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:55.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.381+0000 7fd247bf0640 1 -- 172.21.15.53:0/813584692 shutdown_connections 2024-09-19T06:09:55.838 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:54.381+0000 7fd247bf0640 1 -- 172.21.15.53:0/813584692 wait complete. 2024-09-19T06:09:55.838 INFO:teuthology.orchestra.run.smithi053.stdout:Waiting for the mgr to restart... 2024-09-19T06:09:55.838 INFO:teuthology.orchestra.run.smithi053.stdout:Waiting for mgr epoch 5... 2024-09-19T06:09:58.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:58 smithi053 ceph-mon[27968]: Active manager daemon smithi053.trakvg restarted 2024-09-19T06:09:58.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:58 smithi053 ceph-mon[27968]: Activating manager daemon smithi053.trakvg 2024-09-19T06:09:58.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:58 smithi053 ceph-mon[27968]: osdmap e2: 0 total, 0 up, 0 in 2024-09-19T06:09:58.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:58 smithi053 ceph-mon[27968]: mgrmap e6: smithi053.trakvg(active, starting, since 0.00325823s) 2024-09-19T06:09:58.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:58 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata", "id": "smithi053"}]: dispatch 2024-09-19T06:09:58.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:58 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mgr metadata", "who": "smithi053.trakvg", "id": "smithi053.trakvg"}]: dispatch 2024-09-19T06:09:58.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:58 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T06:09:58.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:58 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T06:09:58.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:58 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T06:09:58.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:58 smithi053 ceph-mon[27968]: Manager daemon smithi053.trakvg is now available 2024-09-19T06:09:58.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:58 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' 2024-09-19T06:09:58.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:58 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' 2024-09-19T06:09:58.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:58 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:09:58.607 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:58 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:09:58.607 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:58 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi053.trakvg/mirror_snapshot_schedule"}]: dispatch 2024-09-19T06:09:58.607 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:09:58 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi053.trakvg/trash_purge_schedule"}]: dispatch 2024-09-19T06:09:59.880 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout { 2024-09-19T06:09:59.880 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-19T06:09:59.880 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T06:09:59.880 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout } 2024-09-19T06:09:59.880 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.223+0000 7f05c371c640 1 Processor -- start 2024-09-19T06:09:59.880 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.223+0000 7f05c371c640 1 -- start start 2024-09-19T06:09:59.880 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.223+0000 7f05c371c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05bc107550 0x7f05bc109940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:59.880 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.223+0000 7f05c371c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc07a730 con 0x7f05bc107550 2024-09-19T06:09:59.880 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.224+0000 7f05c14b8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05bc107550 0x7f05bc109940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:59.880 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.224+0000 7f05c14b8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05bc107550 0x7f05bc109940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41754/0 (socket says 172.21.15.53:41754) 2024-09-19T06:09:59.881 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.224+0000 7f05c14b8640 1 -- 172.21.15.53:0/928342809 learned_addr learned my addr 172.21.15.53:0/928342809 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:09:59.881 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.224+0000 7f05c14b8640 1 -- 172.21.15.53:0/928342809 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05bc109e80 con 0x7f05bc107550 2024-09-19T06:09:59.881 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.225+0000 7f05c14b8640 1 --2- 172.21.15.53:0/928342809 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05bc107550 0x7f05bc109940 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f05b8009820 tx=0x7f05b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545ecab3e522112c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:59.881 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.225+0000 7f05b3fff640 1 -- 172.21.15.53:0/928342809 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f05b8037070 con 0x7f05bc107550 2024-09-19T06:09:59.881 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.225+0000 7f05b3fff640 1 -- 172.21.15.53:0/928342809 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f05b802ad40 con 0x7f05bc107550 2024-09-19T06:09:59.881 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.225+0000 7f05b3fff640 1 -- 172.21.15.53:0/928342809 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f05b8030670 con 0x7f05bc107550 2024-09-19T06:09:59.881 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.226+0000 7f05c371c640 1 -- 172.21.15.53:0/928342809 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05bc107550 msgr2=0x7f05bc109940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:59.881 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.226+0000 7f05c371c640 1 --2- 172.21.15.53:0/928342809 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05bc107550 0x7f05bc109940 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f05b8009820 tx=0x7f05b802a270 comp rx=0 tx=0).stop 2024-09-19T06:09:59.881 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.226+0000 7f05c371c640 1 -- 172.21.15.53:0/928342809 shutdown_connections 2024-09-19T06:09:59.881 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.226+0000 7f05c371c640 1 --2- 172.21.15.53:0/928342809 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05bc107550 0x7f05bc109940 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:59.881 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.226+0000 7f05c371c640 1 -- 172.21.15.53:0/928342809 >> 172.21.15.53:0/928342809 conn(0x7f05bc101150 msgr2=0x7f05bc1035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:59.881 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.226+0000 7f05c371c640 1 -- 172.21.15.53:0/928342809 shutdown_connections 2024-09-19T06:09:59.881 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.226+0000 7f05c371c640 1 -- 172.21.15.53:0/928342809 wait complete. 2024-09-19T06:09:59.882 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.227+0000 7f05c371c640 1 Processor -- start 2024-09-19T06:09:59.882 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.227+0000 7f05c371c640 1 -- start start 2024-09-19T06:09:59.882 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.227+0000 7f05c371c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05bc107550 0x7f05bc19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:59.882 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.227+0000 7f05c371c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc10aec0 con 0x7f05bc107550 2024-09-19T06:09:59.882 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.227+0000 7f05c14b8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05bc107550 0x7f05bc19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:59.882 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.227+0000 7f05c14b8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05bc107550 0x7f05bc19aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41760/0 (socket says 172.21.15.53:41760) 2024-09-19T06:09:59.882 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.227+0000 7f05c14b8640 1 -- 172.21.15.53:0/3701000564 learned_addr learned my addr 172.21.15.53:0/3701000564 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:09:59.882 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.227+0000 7f05c14b8640 1 -- 172.21.15.53:0/3701000564 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05bc19af80 con 0x7f05bc107550 2024-09-19T06:09:59.882 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.227+0000 7f05c14b8640 1 --2- 172.21.15.53:0/3701000564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05bc107550 0x7f05bc19aa40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f05b8030380 tx=0x7f05b8032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:59.882 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.228+0000 7f05b27fc640 1 -- 172.21.15.53:0/3701000564 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f05b8043070 con 0x7f05bc107550 2024-09-19T06:09:59.882 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.228+0000 7f05b27fc640 1 -- 172.21.15.53:0/3701000564 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f05b803e480 con 0x7f05bc107550 2024-09-19T06:09:59.882 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.228+0000 7f05c371c640 1 -- 172.21.15.53:0/3701000564 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05bc19b210 con 0x7f05bc107550 2024-09-19T06:09:59.882 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.228+0000 7f05b27fc640 1 -- 172.21.15.53:0/3701000564 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f05b803c510 con 0x7f05bc107550 2024-09-19T06:09:59.882 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.228+0000 7f05c371c640 1 -- 172.21.15.53:0/3701000564 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05bc19b670 con 0x7f05bc107550 2024-09-19T06:09:59.883 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.229+0000 7f05b27fc640 1 -- 172.21.15.53:0/3701000564 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 46476+0+0 (secure 0 0 0) 0x7f05b8046070 con 0x7f05bc107550 2024-09-19T06:09:59.883 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.229+0000 7f05b27fc640 1 --2- 172.21.15.53:0/3701000564 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7f05a803d5f0 0x7f05a803fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:59.883 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.229+0000 7f05b27fc640 1 -- 172.21.15.53:0/3701000564 --> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f05a80401c0 con 0x7f05a803d5f0 2024-09-19T06:09:59.883 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.229+0000 7f05b27fc640 1 -- 172.21.15.53:0/3701000564 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f05b806c000 con 0x7f05bc107550 2024-09-19T06:09:59.883 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.229+0000 7f05c0cb7640 1 -- 172.21.15.53:0/3701000564 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7f05a803d5f0 msgr2=0x7f05a803fab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/475342876 2024-09-19T06:09:59.883 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.229+0000 7f05c0cb7640 1 --2- 172.21.15.53:0/3701000564 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7f05a803d5f0 0x7f05a803fab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T06:09:59.883 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.430+0000 7f05c0cb7640 1 -- 172.21.15.53:0/3701000564 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7f05a803d5f0 msgr2=0x7f05a803fab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/475342876 2024-09-19T06:09:59.883 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.430+0000 7f05c0cb7640 1 --2- 172.21.15.53:0/3701000564 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7f05a803d5f0 0x7f05a803fab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T06:09:59.883 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.831+0000 7f05c0cb7640 1 -- 172.21.15.53:0/3701000564 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7f05a803d5f0 msgr2=0x7f05a803fab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/475342876 2024-09-19T06:09:59.883 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:56.831+0000 7f05c0cb7640 1 --2- 172.21.15.53:0/3701000564 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7f05a803d5f0 0x7f05a803fab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T06:09:59.883 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:57.632+0000 7f05c0cb7640 1 -- 172.21.15.53:0/3701000564 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7f05a803d5f0 msgr2=0x7f05a803fab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/475342876 2024-09-19T06:09:59.883 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:57.632+0000 7f05c0cb7640 1 --2- 172.21.15.53:0/3701000564 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7f05a803d5f0 0x7f05a803fab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T06:09:59.883 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:58.272+0000 7f05b27fc640 1 -- 172.21.15.53:0/3701000564 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mgrmap(e 6) ==== 46243+0+0 (secure 0 0 0) 0x7f05b806ab80 con 0x7f05bc107550 2024-09-19T06:09:59.883 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:58.273+0000 7f05b27fc640 1 -- 172.21.15.53:0/3701000564 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7f05a803d5f0 msgr2=0x7f05a803fab0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T06:09:59.884 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:58.273+0000 7f05b27fc640 1 --2- 172.21.15.53:0/3701000564 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7f05a803d5f0 0x7f05a803fab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:59.884 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:59.274+0000 7f05b27fc640 1 -- 172.21.15.53:0/3701000564 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 7) ==== 46370+0+0 (secure 0 0 0) 0x7f05b8045cf0 con 0x7f05bc107550 2024-09-19T06:09:59.884 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:59.274+0000 7f05b27fc640 1 --2- 172.21.15.53:0/3701000564 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f05a8040f20 0x7f05a8043310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:59.884 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:59.274+0000 7f05b27fc640 1 -- 172.21.15.53:0/3701000564 --> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f05b8039e50 con 0x7f05a8040f20 2024-09-19T06:09:59.884 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:59.274+0000 7f05c0cb7640 1 --2- 172.21.15.53:0/3701000564 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f05a8040f20 0x7f05a8043310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:59.884 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:59.275+0000 7f05c0cb7640 1 --2- 172.21.15.53:0/3701000564 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f05a8040f20 0x7f05a8043310 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f05ac003980 tx=0x7f05ac006e10 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:59.884 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:59.276+0000 7f05b27fc640 1 -- 172.21.15.53:0/3701000564 <== mgr.14118 v2:172.21.15.53:6800/681468213 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f05b8039e50 con 0x7f05a8040f20 2024-09-19T06:09:59.884 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:59.283+0000 7f05c371c640 1 -- 172.21.15.53:0/3701000564 --> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f05bc1079b0 con 0x7f05a8040f20 2024-09-19T06:09:59.884 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:59.283+0000 7f05b27fc640 1 -- 172.21.15.53:0/3701000564 <== mgr.14118 v2:172.21.15.53:6800/681468213 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f05bc1079b0 con 0x7f05a8040f20 2024-09-19T06:09:59.884 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:59.283+0000 7f05c371c640 1 -- 172.21.15.53:0/3701000564 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f05a8040f20 msgr2=0x7f05a8043310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:59.884 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:59.283+0000 7f05c371c640 1 --2- 172.21.15.53:0/3701000564 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f05a8040f20 0x7f05a8043310 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f05ac003980 tx=0x7f05ac006e10 comp rx=0 tx=0).stop 2024-09-19T06:09:59.884 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:59.283+0000 7f05c371c640 1 -- 172.21.15.53:0/3701000564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05bc107550 msgr2=0x7f05bc19aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:59.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:59.283+0000 7f05c371c640 1 --2- 172.21.15.53:0/3701000564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05bc107550 0x7f05bc19aa40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f05b8030380 tx=0x7f05b8032670 comp rx=0 tx=0).stop 2024-09-19T06:09:59.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:59.283+0000 7f05c371c640 1 -- 172.21.15.53:0/3701000564 shutdown_connections 2024-09-19T06:09:59.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:59.283+0000 7f05c371c640 1 --2- 172.21.15.53:0/3701000564 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f05a8040f20 0x7f05a8043310 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:59.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:59.283+0000 7f05c371c640 1 --2- 172.21.15.53:0/3701000564 >> [v2:172.21.15.53:6800/475342876,v1:172.21.15.53:6801/475342876] conn(0x7f05a803d5f0 0x7f05a803fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:59.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:59.283+0000 7f05c371c640 1 --2- 172.21.15.53:0/3701000564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05bc107550 0x7f05bc19aa40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:59.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:59.283+0000 7f05c371c640 1 -- 172.21.15.53:0/3701000564 >> 172.21.15.53:0/3701000564 conn(0x7f05bc101150 msgr2=0x7f05bc101d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:59.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:59.283+0000 7f05c371c640 1 -- 172.21.15.53:0/3701000564 shutdown_connections 2024-09-19T06:09:59.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:09:59.284+0000 7f05c371c640 1 -- 172.21.15.53:0/3701000564 wait complete. 2024-09-19T06:09:59.885 INFO:teuthology.orchestra.run.smithi053.stdout:mgr epoch 5 is available 2024-09-19T06:09:59.885 INFO:teuthology.orchestra.run.smithi053.stdout:Setting orchestrator backend to cephadm... 2024-09-19T06:10:00.467 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:00 smithi053 ceph-mon[27968]: Found migration_current of "None". Setting to last migration. 2024-09-19T06:10:00.467 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:00 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:00.467 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:00 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:00.467 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:00 smithi053 ceph-mon[27968]: mgrmap e7: smithi053.trakvg(active, since 1.00543s) 2024-09-19T06:10:01.042 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.261+0000 7fb3710c5640 1 Processor -- start 2024-09-19T06:10:01.043 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.261+0000 7fb3710c5640 1 -- start start 2024-09-19T06:10:01.043 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.261+0000 7fb3710c5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb36c107540 0x7fb36c109930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:01.043 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.261+0000 7fb3710c5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb36c07a730 con 0x7fb36c107540 2024-09-19T06:10:01.043 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.262+0000 7fb36ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb36c107540 0x7fb36c109930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:01.043 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.262+0000 7fb36ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb36c107540 0x7fb36c109930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41842/0 (socket says 172.21.15.53:41842) 2024-09-19T06:10:01.043 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.262+0000 7fb36ad9d640 1 -- 172.21.15.53:0/2285671822 learned_addr learned my addr 172.21.15.53:0/2285671822 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:01.043 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.262+0000 7fb36ad9d640 1 -- 172.21.15.53:0/2285671822 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb36c109e70 con 0x7fb36c107540 2024-09-19T06:10:01.043 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.263+0000 7fb36ad9d640 1 --2- 172.21.15.53:0/2285671822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb36c107540 0x7fb36c109930 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb35c009820 tx=0x7fb35c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0df00ad9dd1acdb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:01.043 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.263+0000 7fb36a59c640 1 -- 172.21.15.53:0/2285671822 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fb35c037070 con 0x7fb36c107540 2024-09-19T06:10:01.043 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.263+0000 7fb36a59c640 1 -- 172.21.15.53:0/2285671822 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb35c02ad40 con 0x7fb36c107540 2024-09-19T06:10:01.043 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.263+0000 7fb36a59c640 1 -- 172.21.15.53:0/2285671822 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fb35c030670 con 0x7fb36c107540 2024-09-19T06:10:01.044 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.263+0000 7fb3710c5640 1 -- 172.21.15.53:0/2285671822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb36c107540 msgr2=0x7fb36c109930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:01.044 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.263+0000 7fb3710c5640 1 --2- 172.21.15.53:0/2285671822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb36c107540 0x7fb36c109930 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb35c009820 tx=0x7fb35c02a270 comp rx=0 tx=0).stop 2024-09-19T06:10:01.044 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.264+0000 7fb3710c5640 1 -- 172.21.15.53:0/2285671822 shutdown_connections 2024-09-19T06:10:01.044 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.264+0000 7fb3710c5640 1 --2- 172.21.15.53:0/2285671822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb36c107540 0x7fb36c109930 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:01.044 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.264+0000 7fb3710c5640 1 -- 172.21.15.53:0/2285671822 >> 172.21.15.53:0/2285671822 conn(0x7fb36c1011a0 msgr2=0x7fb36c1035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:01.044 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.264+0000 7fb3710c5640 1 -- 172.21.15.53:0/2285671822 shutdown_connections 2024-09-19T06:10:01.044 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.264+0000 7fb3710c5640 1 -- 172.21.15.53:0/2285671822 wait complete. 2024-09-19T06:10:01.044 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.264+0000 7fb3710c5640 1 Processor -- start 2024-09-19T06:10:01.044 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.264+0000 7fb3710c5640 1 -- start start 2024-09-19T06:10:01.044 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.265+0000 7fb3710c5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb36c107540 0x7fb36c106dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:01.044 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.265+0000 7fb3710c5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb36c10aeb0 con 0x7fb36c107540 2024-09-19T06:10:01.044 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.265+0000 7fb36ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb36c107540 0x7fb36c106dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:01.045 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.265+0000 7fb36ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb36c107540 0x7fb36c106dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41858/0 (socket says 172.21.15.53:41858) 2024-09-19T06:10:01.045 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.265+0000 7fb36ad9d640 1 -- 172.21.15.53:0/1013606598 learned_addr learned my addr 172.21.15.53:0/1013606598 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:01.045 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.265+0000 7fb36ad9d640 1 -- 172.21.15.53:0/1013606598 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb36c107300 con 0x7fb36c107540 2024-09-19T06:10:01.045 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.265+0000 7fb36ad9d640 1 --2- 172.21.15.53:0/1013606598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb36c107540 0x7fb36c106dc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb35c030380 tx=0x7fb35c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:01.045 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.265+0000 7fb368d99640 1 -- 172.21.15.53:0/1013606598 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fb35c043070 con 0x7fb36c107540 2024-09-19T06:10:01.045 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.265+0000 7fb368d99640 1 -- 172.21.15.53:0/1013606598 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb35c03e480 con 0x7fb36c107540 2024-09-19T06:10:01.045 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.265+0000 7fb3710c5640 1 -- 172.21.15.53:0/1013606598 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb36c1054f0 con 0x7fb36c107540 2024-09-19T06:10:01.045 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.265+0000 7fb368d99640 1 -- 172.21.15.53:0/1013606598 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fb35c03c510 con 0x7fb36c107540 2024-09-19T06:10:01.045 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.265+0000 7fb3710c5640 1 -- 172.21.15.53:0/1013606598 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb36c105a30 con 0x7fb36c107540 2024-09-19T06:10:01.045 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.266+0000 7fb3710c5640 1 -- 172.21.15.53:0/1013606598 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb36c1079a0 con 0x7fb36c107540 2024-09-19T06:10:01.045 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.267+0000 7fb368d99640 1 -- 172.21.15.53:0/1013606598 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 7) ==== 46370+0+0 (secure 0 0 0) 0x7fb35c046070 con 0x7fb36c107540 2024-09-19T06:10:01.045 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.267+0000 7fb368d99640 1 --2- 172.21.15.53:0/1013606598 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fb3580390d0 0x7fb35803b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:01.045 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.267+0000 7fb368d99640 1 -- 172.21.15.53:0/1013606598 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb35c06bb10 con 0x7fb36c107540 2024-09-19T06:10:01.045 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.267+0000 7fb363fff640 1 --2- 172.21.15.53:0/1013606598 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fb3580390d0 0x7fb35803b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:01.046 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.267+0000 7fb363fff640 1 --2- 172.21.15.53:0/1013606598 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fb3580390d0 0x7fb35803b590 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb3540098c0 tx=0x7fb354006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:01.046 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.276+0000 7fb368d99640 1 -- 172.21.15.53:0/1013606598 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb35c046360 con 0x7fb36c107540 2024-09-19T06:10:01.046 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.465+0000 7fb3710c5640 1 -- 172.21.15.53:0/1013606598 --> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fb36c106380 con 0x7fb3580390d0 2024-09-19T06:10:01.046 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.470+0000 7fb368d99640 1 -- 172.21.15.53:0/1013606598 <== mgr.14118 v2:172.21.15.53:6800/681468213 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb36c106380 con 0x7fb3580390d0 2024-09-19T06:10:01.046 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.472+0000 7fb3710c5640 1 -- 172.21.15.53:0/1013606598 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fb3580390d0 msgr2=0x7fb35803b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:01.046 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.472+0000 7fb3710c5640 1 --2- 172.21.15.53:0/1013606598 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fb3580390d0 0x7fb35803b590 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb3540098c0 tx=0x7fb354006db0 comp rx=0 tx=0).stop 2024-09-19T06:10:01.046 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.472+0000 7fb3710c5640 1 -- 172.21.15.53:0/1013606598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb36c107540 msgr2=0x7fb36c106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:01.046 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.472+0000 7fb3710c5640 1 --2- 172.21.15.53:0/1013606598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb36c107540 0x7fb36c106dc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb35c030380 tx=0x7fb35c032670 comp rx=0 tx=0).stop 2024-09-19T06:10:01.046 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.473+0000 7fb3710c5640 1 -- 172.21.15.53:0/1013606598 shutdown_connections 2024-09-19T06:10:01.046 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.473+0000 7fb3710c5640 1 --2- 172.21.15.53:0/1013606598 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fb3580390d0 0x7fb35803b590 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:01.046 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.473+0000 7fb3710c5640 1 --2- 172.21.15.53:0/1013606598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb36c107540 0x7fb36c106dc0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:01.046 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.473+0000 7fb3710c5640 1 -- 172.21.15.53:0/1013606598 >> 172.21.15.53:0/1013606598 conn(0x7fb36c1011a0 msgr2=0x7fb36c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:01.046 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.473+0000 7fb3710c5640 1 -- 172.21.15.53:0/1013606598 shutdown_connections 2024-09-19T06:10:01.047 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:00.473+0000 7fb3710c5640 1 -- 172.21.15.53:0/1013606598 wait complete. 2024-09-19T06:10:01.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:01 smithi053 ceph-mon[27968]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-19T06:10:01.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:01 smithi053 ceph-mon[27968]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-19T06:10:01.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:01 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:01.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:01 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:10:01.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:01 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:10:02.250 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:02 smithi053 ceph-mon[27968]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:10:02.250 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:02 smithi053 ceph-mon[27968]: [19/Sep/2024:06:10:00] ENGINE Bus STARTING 2024-09-19T06:10:02.251 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:02 smithi053 ceph-mon[27968]: [19/Sep/2024:06:10:00] ENGINE Serving on https://172.21.15.53:7150 2024-09-19T06:10:02.251 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:02 smithi053 ceph-mon[27968]: [19/Sep/2024:06:10:00] ENGINE Error in HTTPServer.serve 2024-09-19T06:10:02.251 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Traceback (most recent call last): 2024-09-19T06:10:02.251 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T06:10:02.251 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: self._connections.run(self.expiration_interval) 2024-09-19T06:10:02.251 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T06:10:02.251 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: self._run(expiration_interval) 2024-09-19T06:10:02.251 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T06:10:02.251 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T06:10:02.251 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T06:10:02.251 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T06:10:02.251 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T06:10:02.251 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: s = self.context.wrap_socket( 2024-09-19T06:10:02.252 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T06:10:02.252 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: return self.sslsocket_class._create( 2024-09-19T06:10:02.252 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T06:10:02.252 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: self.do_handshake() 2024-09-19T06:10:02.252 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T06:10:02.252 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: self._sslobj.do_handshake() 2024-09-19T06:10:02.252 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T06:10:02.252 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:02 smithi053 ceph-mon[27968]: [19/Sep/2024:06:10:00] ENGINE Serving on http://172.21.15.53:8765 2024-09-19T06:10:02.252 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:02 smithi053 ceph-mon[27968]: [19/Sep/2024:06:10:00] ENGINE Bus STARTED 2024-09-19T06:10:02.252 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:02 smithi053 ceph-mon[27968]: mgrmap e8: smithi053.trakvg(active, since 2s) 2024-09-19T06:10:02.258 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-19T06:10:02.258 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.467+0000 7fb0df1db640 1 Processor -- start 2024-09-19T06:10:02.258 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.467+0000 7fb0df1db640 1 -- start start 2024-09-19T06:10:02.259 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.467+0000 7fb0df1db640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0d807c630 0x7fb0d807aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:02.259 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.467+0000 7fb0df1db640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d807b0e0 con 0x7fb0d807c630 2024-09-19T06:10:02.259 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.467+0000 7fb0dcf77640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0d807c630 0x7fb0d807aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:02.259 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.468+0000 7fb0dcf77640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0d807c630 0x7fb0d807aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50262/0 (socket says 172.21.15.53:50262) 2024-09-19T06:10:02.259 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.468+0000 7fb0dcf77640 1 -- 172.21.15.53:0/819732103 learned_addr learned my addr 172.21.15.53:0/819732103 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:02.259 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.468+0000 7fb0dcf77640 1 -- 172.21.15.53:0/819732103 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d807b260 con 0x7fb0d807c630 2024-09-19T06:10:02.259 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.468+0000 7fb0dcf77640 1 --2- 172.21.15.53:0/819732103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0d807c630 0x7fb0d807aba0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc009820 tx=0x7fb0cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffdc9ac358fa9f99 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:02.259 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.469+0000 7fb0d77fe640 1 -- 172.21.15.53:0/819732103 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fb0cc037070 con 0x7fb0d807c630 2024-09-19T06:10:02.259 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.469+0000 7fb0d77fe640 1 -- 172.21.15.53:0/819732103 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb0cc02ad40 con 0x7fb0d807c630 2024-09-19T06:10:02.259 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.469+0000 7fb0d77fe640 1 -- 172.21.15.53:0/819732103 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fb0cc030670 con 0x7fb0d807c630 2024-09-19T06:10:02.259 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.469+0000 7fb0df1db640 1 -- 172.21.15.53:0/819732103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0d807c630 msgr2=0x7fb0d807aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:02.259 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.469+0000 7fb0df1db640 1 --2- 172.21.15.53:0/819732103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0d807c630 0x7fb0d807aba0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc009820 tx=0x7fb0cc02a270 comp rx=0 tx=0).stop 2024-09-19T06:10:02.259 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.469+0000 7fb0df1db640 1 -- 172.21.15.53:0/819732103 shutdown_connections 2024-09-19T06:10:02.260 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.469+0000 7fb0df1db640 1 --2- 172.21.15.53:0/819732103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0d807c630 0x7fb0d807aba0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:02.260 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.469+0000 7fb0df1db640 1 -- 172.21.15.53:0/819732103 >> 172.21.15.53:0/819732103 conn(0x7fb0d81023c0 msgr2=0x7fb0d81047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:02.260 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.469+0000 7fb0df1db640 1 -- 172.21.15.53:0/819732103 shutdown_connections 2024-09-19T06:10:02.260 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.469+0000 7fb0df1db640 1 -- 172.21.15.53:0/819732103 wait complete. 2024-09-19T06:10:02.260 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.470+0000 7fb0df1db640 1 Processor -- start 2024-09-19T06:10:02.260 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.470+0000 7fb0df1db640 1 -- start start 2024-09-19T06:10:02.260 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.470+0000 7fb0df1db640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0d807c630 0x7fb0d819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:02.260 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.470+0000 7fb0df1db640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d810cff0 con 0x7fb0d807c630 2024-09-19T06:10:02.260 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.470+0000 7fb0dcf77640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0d807c630 0x7fb0d819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:02.260 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.470+0000 7fb0dcf77640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0d807c630 0x7fb0d819cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50266/0 (socket says 172.21.15.53:50266) 2024-09-19T06:10:02.260 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.470+0000 7fb0dcf77640 1 -- 172.21.15.53:0/15899267 learned_addr learned my addr 172.21.15.53:0/15899267 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:02.260 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.470+0000 7fb0dcf77640 1 -- 172.21.15.53:0/15899267 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d819d110 con 0x7fb0d807c630 2024-09-19T06:10:02.260 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.471+0000 7fb0dcf77640 1 --2- 172.21.15.53:0/15899267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0d807c630 0x7fb0d819cbd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc000c00 tx=0x7fb0cc032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:02.261 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.471+0000 7fb0d5ffb640 1 -- 172.21.15.53:0/15899267 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fb0cc043070 con 0x7fb0d807c630 2024-09-19T06:10:02.261 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.471+0000 7fb0df1db640 1 -- 172.21.15.53:0/15899267 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d819d3a0 con 0x7fb0d807c630 2024-09-19T06:10:02.261 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.471+0000 7fb0d5ffb640 1 -- 172.21.15.53:0/15899267 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb0cc03e480 con 0x7fb0d807c630 2024-09-19T06:10:02.261 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.471+0000 7fb0d5ffb640 1 -- 172.21.15.53:0/15899267 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fb0cc03c530 con 0x7fb0d807c630 2024-09-19T06:10:02.261 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.471+0000 7fb0df1db640 1 -- 172.21.15.53:0/15899267 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d81a0090 con 0x7fb0d807c630 2024-09-19T06:10:02.261 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.472+0000 7fb0d5ffb640 1 -- 172.21.15.53:0/15899267 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) ==== 46476+0+0 (secure 0 0 0) 0x7fb0cc046070 con 0x7fb0d807c630 2024-09-19T06:10:02.261 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.472+0000 7fb0df1db640 1 -- 172.21.15.53:0/15899267 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0d807ac20 con 0x7fb0d807c630 2024-09-19T06:10:02.261 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.472+0000 7fb0d5ffb640 1 --2- 172.21.15.53:0/15899267 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fb0c80391e0 0x7fb0c803b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:02.261 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.472+0000 7fb0d5ffb640 1 -- 172.21.15.53:0/15899267 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb0cc06bbd0 con 0x7fb0d807c630 2024-09-19T06:10:02.261 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.472+0000 7fb0d7fff640 1 --2- 172.21.15.53:0/15899267 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fb0c80391e0 0x7fb0c803b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:02.262 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.473+0000 7fb0d7fff640 1 --2- 172.21.15.53:0/15899267 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fb0c80391e0 0x7fb0c803b6a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40098c0 tx=0x7fb0c4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:02.262 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.477+0000 7fb0d5ffb640 1 -- 172.21.15.53:0/15899267 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb0cc046360 con 0x7fb0d807c630 2024-09-19T06:10:02.262 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.655+0000 7fb0df1db640 1 -- 172.21.15.53:0/15899267 --> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fb0d8103600 con 0x7fb0c80391e0 2024-09-19T06:10:02.262 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.656+0000 7fb0d5ffb640 1 -- 172.21.15.53:0/15899267 <== mgr.14118 v2:172.21.15.53:6800/681468213 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fb0d8103600 con 0x7fb0c80391e0 2024-09-19T06:10:02.262 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.658+0000 7fb0df1db640 1 -- 172.21.15.53:0/15899267 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fb0c80391e0 msgr2=0x7fb0c803b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:02.262 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.658+0000 7fb0df1db640 1 --2- 172.21.15.53:0/15899267 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fb0c80391e0 0x7fb0c803b6a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40098c0 tx=0x7fb0c4006db0 comp rx=0 tx=0).stop 2024-09-19T06:10:02.262 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.658+0000 7fb0df1db640 1 -- 172.21.15.53:0/15899267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0d807c630 msgr2=0x7fb0d819cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:02.262 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.658+0000 7fb0df1db640 1 --2- 172.21.15.53:0/15899267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0d807c630 0x7fb0d819cbd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc000c00 tx=0x7fb0cc032670 comp rx=0 tx=0).stop 2024-09-19T06:10:02.262 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.658+0000 7fb0df1db640 1 -- 172.21.15.53:0/15899267 shutdown_connections 2024-09-19T06:10:02.262 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.658+0000 7fb0df1db640 1 --2- 172.21.15.53:0/15899267 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fb0c80391e0 0x7fb0c803b6a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:02.262 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.658+0000 7fb0df1db640 1 --2- 172.21.15.53:0/15899267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0d807c630 0x7fb0d819cbd0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:02.262 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.658+0000 7fb0df1db640 1 -- 172.21.15.53:0/15899267 >> 172.21.15.53:0/15899267 conn(0x7fb0d81023c0 msgr2=0x7fb0d8102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:02.262 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.658+0000 7fb0df1db640 1 -- 172.21.15.53:0/15899267 shutdown_connections 2024-09-19T06:10:02.263 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:01.658+0000 7fb0df1db640 1 -- 172.21.15.53:0/15899267 wait complete. 2024-09-19T06:10:02.263 INFO:teuthology.orchestra.run.smithi053.stdout:Generating ssh key... 2024-09-19T06:10:03.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:03 smithi053 ceph-mon[27968]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:10:03.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:03 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:03.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:03 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:03.623 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.659+0000 7efe09065640 1 Processor -- start 2024-09-19T06:10:03.623 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.659+0000 7efe09065640 1 -- start start 2024-09-19T06:10:03.623 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.659+0000 7efe09065640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe041053b0 0x7efe041057b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:03.623 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.659+0000 7efe09065640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efe0407a6b0 con 0x7efe041053b0 2024-09-19T06:10:03.623 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.660+0000 7efe02d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe041053b0 0x7efe041057b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:03.623 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.660+0000 7efe02d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe041053b0 0x7efe041057b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50270/0 (socket says 172.21.15.53:50270) 2024-09-19T06:10:03.623 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.660+0000 7efe02d9d640 1 -- 172.21.15.53:0/3833046680 learned_addr learned my addr 172.21.15.53:0/3833046680 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:03.624 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.660+0000 7efe02d9d640 1 -- 172.21.15.53:0/3833046680 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe04105cf0 con 0x7efe041053b0 2024-09-19T06:10:03.624 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.661+0000 7efe02d9d640 1 --2- 172.21.15.53:0/3833046680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe041053b0 0x7efe041057b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efdf4009820 tx=0x7efdf402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1edf6272bbf3af90 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:03.624 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.661+0000 7efe01d9b640 1 -- 172.21.15.53:0/3833046680 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7efdf4037070 con 0x7efe041053b0 2024-09-19T06:10:03.624 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.661+0000 7efe01d9b640 1 -- 172.21.15.53:0/3833046680 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7efdf402ad40 con 0x7efe041053b0 2024-09-19T06:10:03.624 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.661+0000 7efe01d9b640 1 -- 172.21.15.53:0/3833046680 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7efdf4030670 con 0x7efe041053b0 2024-09-19T06:10:03.624 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.661+0000 7efe09065640 1 -- 172.21.15.53:0/3833046680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe041053b0 msgr2=0x7efe041057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:03.624 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.661+0000 7efe09065640 1 --2- 172.21.15.53:0/3833046680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe041053b0 0x7efe041057b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efdf4009820 tx=0x7efdf402a270 comp rx=0 tx=0).stop 2024-09-19T06:10:03.624 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.661+0000 7efe09065640 1 -- 172.21.15.53:0/3833046680 shutdown_connections 2024-09-19T06:10:03.624 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.661+0000 7efe09065640 1 --2- 172.21.15.53:0/3833046680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe041053b0 0x7efe041057b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:03.624 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.661+0000 7efe09065640 1 -- 172.21.15.53:0/3833046680 >> 172.21.15.53:0/3833046680 conn(0x7efe041011a0 msgr2=0x7efe041035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:03.624 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.662+0000 7efe09065640 1 -- 172.21.15.53:0/3833046680 shutdown_connections 2024-09-19T06:10:03.624 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.662+0000 7efe09065640 1 -- 172.21.15.53:0/3833046680 wait complete. 2024-09-19T06:10:03.624 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.662+0000 7efe09065640 1 Processor -- start 2024-09-19T06:10:03.625 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.663+0000 7efe09065640 1 -- start start 2024-09-19T06:10:03.625 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.663+0000 7efe09065640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe041053b0 0x7efe0419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:03.625 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.663+0000 7efe09065640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efe04106d90 con 0x7efe041053b0 2024-09-19T06:10:03.625 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.663+0000 7efe02d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe041053b0 0x7efe0419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:03.625 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.663+0000 7efe02d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe041053b0 0x7efe0419cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50286/0 (socket says 172.21.15.53:50286) 2024-09-19T06:10:03.625 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.663+0000 7efe02d9d640 1 -- 172.21.15.53:0/2879271289 learned_addr learned my addr 172.21.15.53:0/2879271289 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:03.625 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.663+0000 7efe02d9d640 1 -- 172.21.15.53:0/2879271289 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe0419d160 con 0x7efe041053b0 2024-09-19T06:10:03.625 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.663+0000 7efe02d9d640 1 --2- 172.21.15.53:0/2879271289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe041053b0 0x7efe0419cc20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efdf4000c00 tx=0x7efdf4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:03.625 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.664+0000 7efdebfff640 1 -- 172.21.15.53:0/2879271289 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7efdf4043070 con 0x7efe041053b0 2024-09-19T06:10:03.625 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.664+0000 7efdebfff640 1 -- 172.21.15.53:0/2879271289 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7efdf403e480 con 0x7efe041053b0 2024-09-19T06:10:03.625 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.664+0000 7efe09065640 1 -- 172.21.15.53:0/2879271289 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe0419d3f0 con 0x7efe041053b0 2024-09-19T06:10:03.625 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.664+0000 7efdebfff640 1 -- 172.21.15.53:0/2879271289 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7efdf403c530 con 0x7efe041053b0 2024-09-19T06:10:03.625 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.664+0000 7efe09065640 1 -- 172.21.15.53:0/2879271289 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe041a00e0 con 0x7efe041053b0 2024-09-19T06:10:03.626 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.665+0000 7efdebfff640 1 -- 172.21.15.53:0/2879271289 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) ==== 46476+0+0 (secure 0 0 0) 0x7efdf4046070 con 0x7efe041053b0 2024-09-19T06:10:03.626 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.665+0000 7efe09065640 1 -- 172.21.15.53:0/2879271289 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe041057b0 con 0x7efe041053b0 2024-09-19T06:10:03.626 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.665+0000 7efdebfff640 1 --2- 172.21.15.53:0/2879271289 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7efdec0391e0 0x7efdec03b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:03.626 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.665+0000 7efdebfff640 1 -- 172.21.15.53:0/2879271289 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efdf406bd10 con 0x7efe041053b0 2024-09-19T06:10:03.626 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.665+0000 7efe0259c640 1 --2- 172.21.15.53:0/2879271289 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7efdec0391e0 0x7efdec03b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:03.626 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.665+0000 7efe0259c640 1 --2- 172.21.15.53:0/2879271289 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7efdec0391e0 0x7efdec03b6a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efdf80098c0 tx=0x7efdf8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:03.626 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.670+0000 7efdebfff640 1 -- 172.21.15.53:0/2879271289 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efdf406ba70 con 0x7efe041053b0 2024-09-19T06:10:03.626 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.844+0000 7efe09065640 1 -- 172.21.15.53:0/2879271289 --> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7efe0402dab0 con 0x7efdec0391e0 2024-09-19T06:10:03.626 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.992+0000 7efdebfff640 1 -- 172.21.15.53:0/2879271289 <== mgr.14118 v2:172.21.15.53:6800/681468213 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efe0402dab0 con 0x7efdec0391e0 2024-09-19T06:10:03.626 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.994+0000 7efe09065640 1 -- 172.21.15.53:0/2879271289 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7efdec0391e0 msgr2=0x7efdec03b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:03.626 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.994+0000 7efe09065640 1 --2- 172.21.15.53:0/2879271289 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7efdec0391e0 0x7efdec03b6a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efdf80098c0 tx=0x7efdf8006db0 comp rx=0 tx=0).stop 2024-09-19T06:10:03.626 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.994+0000 7efe09065640 1 -- 172.21.15.53:0/2879271289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe041053b0 msgr2=0x7efe0419cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:03.626 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.994+0000 7efe09065640 1 --2- 172.21.15.53:0/2879271289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe041053b0 0x7efe0419cc20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efdf4000c00 tx=0x7efdf4032670 comp rx=0 tx=0).stop 2024-09-19T06:10:03.627 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.994+0000 7efe09065640 1 -- 172.21.15.53:0/2879271289 shutdown_connections 2024-09-19T06:10:03.627 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.994+0000 7efe09065640 1 --2- 172.21.15.53:0/2879271289 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7efdec0391e0 0x7efdec03b6a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:03.627 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.994+0000 7efe09065640 1 --2- 172.21.15.53:0/2879271289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe041053b0 0x7efe0419cc20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:03.627 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.994+0000 7efe09065640 1 -- 172.21.15.53:0/2879271289 >> 172.21.15.53:0/2879271289 conn(0x7efe041011a0 msgr2=0x7efe041938c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:03.627 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.994+0000 7efe09065640 1 -- 172.21.15.53:0/2879271289 shutdown_connections 2024-09-19T06:10:03.627 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:02.994+0000 7efe09065640 1 -- 172.21.15.53:0/2879271289 wait complete. 2024-09-19T06:10:04.184 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:04 smithi053 ceph-mon[27968]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:10:04.184 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:04 smithi053 ceph-mon[27968]: Generating ssh key... 2024-09-19T06:10:04.721 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBb9VgqIg+Ix/qTJ2Gp8aKvxKYTwfZu1J68PwIDlf1t856NHsgy5gxsF+JDUdg6NG4oLn+1aH/HMb/RGHTToTwoliFX7nKpclWmXUByg3eBlQc4vk9Pj54eHOTe3bt27E50vWmxYuWCgpb/AW43dHg1n12MdzaDN0xvXNq4EP7pSxQX+Wb6nXhXHbsBjRTi/ti/O4JGdDf4azQ8xjjTkREOchIFh97RgsJEiR/6bvXWpoATQS4YRn4WyNMPCqC11ibwagnHmVDiZ9yNaZg1lcGmon+K0BuLbtq59SrgQ/uHsO9JzhyRpPV8Hlsdj2/l/IrOMIC5HXNvV5K9209JVQ6JCK59t0otmefAcyxNaH8T6ifCzj7O/GitUZDi6JmBuPsTuGd5kdMKrOYWHGKvCYEvB1OR8L5rDYVZtmrDvGqLk3Nr5uu+4LRZekzflPdr9MoVX90qMaaqZLZxL0u91P1bGOOmrKDdX58TB1zNcQjxjUOBfKI5ZqhfS+xEVNoB5E= ceph-9f2876d0-764d-11ef-bceb-c7b262605968 2024-09-19T06:10:04.721 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.994+0000 7fe9144bc640 1 Processor -- start 2024-09-19T06:10:04.721 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.994+0000 7fe9144bc640 1 -- start start 2024-09-19T06:10:04.721 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.994+0000 7fe9144bc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe90c106ac0 0x7fe90c106ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:04.721 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.994+0000 7fe9144bc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe90c107400 con 0x7fe90c106ac0 2024-09-19T06:10:04.722 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.995+0000 7fe912258640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe90c106ac0 0x7fe90c106ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:04.722 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.995+0000 7fe912258640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe90c106ac0 0x7fe90c106ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50300/0 (socket says 172.21.15.53:50300) 2024-09-19T06:10:04.722 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.995+0000 7fe912258640 1 -- 172.21.15.53:0/1548900780 learned_addr learned my addr 172.21.15.53:0/1548900780 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:04.722 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.995+0000 7fe912258640 1 -- 172.21.15.53:0/1548900780 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe90c107580 con 0x7fe90c106ac0 2024-09-19T06:10:04.722 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.996+0000 7fe912258640 1 --2- 172.21.15.53:0/1548900780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe90c106ac0 0x7fe90c106ec0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe908009820 tx=0x7fe90802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e17e3b3ec8006f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:04.722 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.996+0000 7fe911256640 1 -- 172.21.15.53:0/1548900780 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fe908037070 con 0x7fe90c106ac0 2024-09-19T06:10:04.722 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.996+0000 7fe911256640 1 -- 172.21.15.53:0/1548900780 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe90802ad40 con 0x7fe90c106ac0 2024-09-19T06:10:04.722 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.996+0000 7fe911256640 1 -- 172.21.15.53:0/1548900780 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fe908030670 con 0x7fe90c106ac0 2024-09-19T06:10:04.722 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.997+0000 7fe9144bc640 1 -- 172.21.15.53:0/1548900780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe90c106ac0 msgr2=0x7fe90c106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:04.722 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.997+0000 7fe9144bc640 1 --2- 172.21.15.53:0/1548900780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe90c106ac0 0x7fe90c106ec0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe908009820 tx=0x7fe90802a270 comp rx=0 tx=0).stop 2024-09-19T06:10:04.722 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.997+0000 7fe9144bc640 1 -- 172.21.15.53:0/1548900780 shutdown_connections 2024-09-19T06:10:04.722 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.997+0000 7fe9144bc640 1 --2- 172.21.15.53:0/1548900780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe90c106ac0 0x7fe90c106ec0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:04.722 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.997+0000 7fe9144bc640 1 -- 172.21.15.53:0/1548900780 >> 172.21.15.53:0/1548900780 conn(0x7fe90c102330 msgr2=0x7fe90c104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:04.723 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.997+0000 7fe9144bc640 1 -- 172.21.15.53:0/1548900780 shutdown_connections 2024-09-19T06:10:04.723 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.997+0000 7fe9144bc640 1 -- 172.21.15.53:0/1548900780 wait complete. 2024-09-19T06:10:04.723 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.997+0000 7fe9144bc640 1 Processor -- start 2024-09-19T06:10:04.723 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.997+0000 7fe9144bc640 1 -- start start 2024-09-19T06:10:04.723 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.998+0000 7fe9144bc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe90c106ac0 0x7fe90c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:04.723 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.998+0000 7fe9144bc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe90c10af30 con 0x7fe90c106ac0 2024-09-19T06:10:04.723 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.998+0000 7fe912258640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe90c106ac0 0x7fe90c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:04.723 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.998+0000 7fe912258640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe90c106ac0 0x7fe90c19a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50312/0 (socket says 172.21.15.53:50312) 2024-09-19T06:10:04.723 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.998+0000 7fe912258640 1 -- 172.21.15.53:0/3025434315 learned_addr learned my addr 172.21.15.53:0/3025434315 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:04.723 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.998+0000 7fe912258640 1 -- 172.21.15.53:0/3025434315 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe90c19af30 con 0x7fe90c106ac0 2024-09-19T06:10:04.723 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.998+0000 7fe912258640 1 --2- 172.21.15.53:0/3025434315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe90c106ac0 0x7fe90c19a9f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe908000c00 tx=0x7fe908032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:04.723 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.998+0000 7fe9037fe640 1 -- 172.21.15.53:0/3025434315 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fe908043070 con 0x7fe90c106ac0 2024-09-19T06:10:04.723 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.998+0000 7fe9037fe640 1 -- 172.21.15.53:0/3025434315 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe90803e480 con 0x7fe90c106ac0 2024-09-19T06:10:04.723 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.998+0000 7fe9144bc640 1 -- 172.21.15.53:0/3025434315 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe90c19b1c0 con 0x7fe90c106ac0 2024-09-19T06:10:04.724 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.999+0000 7fe9037fe640 1 -- 172.21.15.53:0/3025434315 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fe90803c530 con 0x7fe90c106ac0 2024-09-19T06:10:04.724 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:03.999+0000 7fe9144bc640 1 -- 172.21.15.53:0/3025434315 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe90c19deb0 con 0x7fe90c106ac0 2024-09-19T06:10:04.724 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:04.000+0000 7fe9037fe640 1 -- 172.21.15.53:0/3025434315 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) ==== 46476+0+0 (secure 0 0 0) 0x7fe908046070 con 0x7fe90c106ac0 2024-09-19T06:10:04.724 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:04.000+0000 7fe9144bc640 1 -- 172.21.15.53:0/3025434315 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe90c10b0b0 con 0x7fe90c106ac0 2024-09-19T06:10:04.724 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:04.000+0000 7fe9037fe640 1 --2- 172.21.15.53:0/3025434315 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fe8f80391e0 0x7fe8f803b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:04.724 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:04.000+0000 7fe9037fe640 1 -- 172.21.15.53:0/3025434315 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe90806bbd0 con 0x7fe90c106ac0 2024-09-19T06:10:04.724 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:04.000+0000 7fe911a57640 1 --2- 172.21.15.53:0/3025434315 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fe8f80391e0 0x7fe8f803b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:04.724 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:04.001+0000 7fe911a57640 1 --2- 172.21.15.53:0/3025434315 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fe8f80391e0 0x7fe8f803b6a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc0098c0 tx=0x7fe8fc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:04.724 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:04.005+0000 7fe9037fe640 1 -- 172.21.15.53:0/3025434315 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe90806b930 con 0x7fe90c106ac0 2024-09-19T06:10:04.724 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:04.183+0000 7fe9144bc640 1 -- 172.21.15.53:0/3025434315 --> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fe90c1035d0 con 0x7fe8f80391e0 2024-09-19T06:10:04.724 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:04.184+0000 7fe9037fe640 1 -- 172.21.15.53:0/3025434315 <== mgr.14118 v2:172.21.15.53:6800/681468213 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fe90c1035d0 con 0x7fe8f80391e0 2024-09-19T06:10:04.724 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:04.186+0000 7fe9144bc640 1 -- 172.21.15.53:0/3025434315 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fe8f80391e0 msgr2=0x7fe8f803b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:04.724 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:04.186+0000 7fe9144bc640 1 --2- 172.21.15.53:0/3025434315 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fe8f80391e0 0x7fe8f803b6a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc0098c0 tx=0x7fe8fc006db0 comp rx=0 tx=0).stop 2024-09-19T06:10:04.724 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:04.186+0000 7fe9144bc640 1 -- 172.21.15.53:0/3025434315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe90c106ac0 msgr2=0x7fe90c19a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:04.725 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:04.186+0000 7fe9144bc640 1 --2- 172.21.15.53:0/3025434315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe90c106ac0 0x7fe90c19a9f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe908000c00 tx=0x7fe908032670 comp rx=0 tx=0).stop 2024-09-19T06:10:04.725 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:04.187+0000 7fe9144bc640 1 -- 172.21.15.53:0/3025434315 shutdown_connections 2024-09-19T06:10:04.725 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:04.187+0000 7fe9144bc640 1 --2- 172.21.15.53:0/3025434315 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fe8f80391e0 0x7fe8f803b6a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:04.725 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:04.187+0000 7fe9144bc640 1 --2- 172.21.15.53:0/3025434315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe90c106ac0 0x7fe90c19a9f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:04.725 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:04.187+0000 7fe9144bc640 1 -- 172.21.15.53:0/3025434315 >> 172.21.15.53:0/3025434315 conn(0x7fe90c102330 msgr2=0x7fe90c102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:04.725 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:04.187+0000 7fe9144bc640 1 -- 172.21.15.53:0/3025434315 shutdown_connections 2024-09-19T06:10:04.725 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:04.187+0000 7fe9144bc640 1 -- 172.21.15.53:0/3025434315 wait complete. 2024-09-19T06:10:04.725 INFO:teuthology.orchestra.run.smithi053.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-19T06:10:04.725 INFO:teuthology.orchestra.run.smithi053.stdout:Adding key to root@localhost authorized_keys... 2024-09-19T06:10:04.726 INFO:teuthology.orchestra.run.smithi053.stdout:Adding host smithi053... 2024-09-19T06:10:05.263 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:05 smithi053 ceph-mon[27968]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:10:06.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:06 smithi053 ceph-mon[27968]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi053", "addr": "172.21.15.53", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:10:07.535 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:07 smithi053 ceph-mon[27968]: Deploying cephadm binary to smithi053 2024-09-19T06:10:09.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout Added host 'smithi053' with addr '172.21.15.53' 2024-09-19T06:10:09.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.074+0000 7f4868489640 1 Processor -- start 2024-09-19T06:10:09.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.074+0000 7f4868489640 1 -- start start 2024-09-19T06:10:09.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.075+0000 7f4868489640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4860106ac0 0x7f4860106ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:09.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.075+0000 7f4868489640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4860107400 con 0x7f4860106ac0 2024-09-19T06:10:09.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.075+0000 7f4866225640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4860106ac0 0x7f4860106ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:09.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.075+0000 7f4866225640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4860106ac0 0x7f4860106ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50318/0 (socket says 172.21.15.53:50318) 2024-09-19T06:10:09.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.075+0000 7f4866225640 1 -- 172.21.15.53:0/2331326375 learned_addr learned my addr 172.21.15.53:0/2331326375 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:09.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.075+0000 7f4866225640 1 -- 172.21.15.53:0/2331326375 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4860107580 con 0x7f4860106ac0 2024-09-19T06:10:09.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.076+0000 7f4866225640 1 --2- 172.21.15.53:0/2331326375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4860106ac0 0x7f4860106ec0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f485c009820 tx=0x7f485c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e55c19586dbd342 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:09.838 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.076+0000 7f4865223640 1 -- 172.21.15.53:0/2331326375 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f485c037070 con 0x7f4860106ac0 2024-09-19T06:10:09.838 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.076+0000 7f4865223640 1 -- 172.21.15.53:0/2331326375 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f485c02ad40 con 0x7f4860106ac0 2024-09-19T06:10:09.838 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.076+0000 7f4865223640 1 -- 172.21.15.53:0/2331326375 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f485c030670 con 0x7f4860106ac0 2024-09-19T06:10:09.838 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.077+0000 7f4868489640 1 -- 172.21.15.53:0/2331326375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4860106ac0 msgr2=0x7f4860106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:09.838 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.077+0000 7f4868489640 1 --2- 172.21.15.53:0/2331326375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4860106ac0 0x7f4860106ec0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f485c009820 tx=0x7f485c02a270 comp rx=0 tx=0).stop 2024-09-19T06:10:09.838 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.077+0000 7f4868489640 1 -- 172.21.15.53:0/2331326375 shutdown_connections 2024-09-19T06:10:09.838 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.077+0000 7f4868489640 1 --2- 172.21.15.53:0/2331326375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4860106ac0 0x7f4860106ec0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:09.838 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.077+0000 7f4868489640 1 -- 172.21.15.53:0/2331326375 >> 172.21.15.53:0/2331326375 conn(0x7f4860102330 msgr2=0x7f4860104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:09.838 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.077+0000 7f4868489640 1 -- 172.21.15.53:0/2331326375 shutdown_connections 2024-09-19T06:10:09.838 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.077+0000 7f4868489640 1 -- 172.21.15.53:0/2331326375 wait complete. 2024-09-19T06:10:09.838 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.077+0000 7f4868489640 1 Processor -- start 2024-09-19T06:10:09.838 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.077+0000 7f4868489640 1 -- start start 2024-09-19T06:10:09.838 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.078+0000 7f4868489640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4860106ac0 0x7f486007c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:09.839 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.078+0000 7f4868489640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f486010af30 con 0x7f4860106ac0 2024-09-19T06:10:09.839 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.078+0000 7f4866225640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4860106ac0 0x7f486007c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:09.839 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.078+0000 7f4866225640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4860106ac0 0x7f486007c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50326/0 (socket says 172.21.15.53:50326) 2024-09-19T06:10:09.839 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.078+0000 7f4866225640 1 -- 172.21.15.53:0/884753210 learned_addr learned my addr 172.21.15.53:0/884753210 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:09.839 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.078+0000 7f4866225640 1 -- 172.21.15.53:0/884753210 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486007c930 con 0x7f4860106ac0 2024-09-19T06:10:09.839 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.078+0000 7f4866225640 1 --2- 172.21.15.53:0/884753210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4860106ac0 0x7f486007c3f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f485c030380 tx=0x7f485c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:09.839 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.078+0000 7f48577fe640 1 -- 172.21.15.53:0/884753210 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f485c043070 con 0x7f4860106ac0 2024-09-19T06:10:09.839 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.078+0000 7f48577fe640 1 -- 172.21.15.53:0/884753210 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f485c03e480 con 0x7f4860106ac0 2024-09-19T06:10:09.839 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.078+0000 7f4868489640 1 -- 172.21.15.53:0/884753210 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f486007ab20 con 0x7f4860106ac0 2024-09-19T06:10:09.839 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.078+0000 7f48577fe640 1 -- 172.21.15.53:0/884753210 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f485c03c510 con 0x7f4860106ac0 2024-09-19T06:10:09.839 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.078+0000 7f4868489640 1 -- 172.21.15.53:0/884753210 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f486007b060 con 0x7f4860106ac0 2024-09-19T06:10:09.839 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.079+0000 7f48577fe640 1 -- 172.21.15.53:0/884753210 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) ==== 46476+0+0 (secure 0 0 0) 0x7f485c046070 con 0x7f4860106ac0 2024-09-19T06:10:09.839 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.079+0000 7f4868489640 1 -- 172.21.15.53:0/884753210 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4860106f40 con 0x7f4860106ac0 2024-09-19T06:10:09.839 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.079+0000 7f48577fe640 1 --2- 172.21.15.53:0/884753210 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f484c039230 0x7f484c03b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:09.840 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.079+0000 7f48577fe640 1 -- 172.21.15.53:0/884753210 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f485c06bc80 con 0x7f4860106ac0 2024-09-19T06:10:09.840 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.080+0000 7f4865a24640 1 --2- 172.21.15.53:0/884753210 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f484c039230 0x7f484c03b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:09.840 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.080+0000 7f4865a24640 1 --2- 172.21.15.53:0/884753210 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f484c039230 0x7f484c03b6f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f48500098c0 tx=0x7f4850006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:09.840 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.084+0000 7f48577fe640 1 -- 172.21.15.53:0/884753210 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f485c046360 con 0x7f4860106ac0 2024-09-19T06:10:09.840 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:05.262+0000 7f4868489640 1 -- 172.21.15.53:0/884753210 --> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi053", "addr": "172.21.15.53", "target": ["mon-mgr", ""]}) -- 0x7f48601035d0 con 0x7f484c039230 2024-09-19T06:10:09.840 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:09.188+0000 7f48577fe640 1 -- 172.21.15.53:0/884753210 <== mgr.14118 v2:172.21.15.53:6800/681468213 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f48601035d0 con 0x7f484c039230 2024-09-19T06:10:09.840 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:09.190+0000 7f4868489640 1 -- 172.21.15.53:0/884753210 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f484c039230 msgr2=0x7f484c03b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:09.840 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:09.191+0000 7f4868489640 1 --2- 172.21.15.53:0/884753210 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f484c039230 0x7f484c03b6f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f48500098c0 tx=0x7f4850006db0 comp rx=0 tx=0).stop 2024-09-19T06:10:09.840 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:09.191+0000 7f4868489640 1 -- 172.21.15.53:0/884753210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4860106ac0 msgr2=0x7f486007c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:09.840 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:09.191+0000 7f4868489640 1 --2- 172.21.15.53:0/884753210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4860106ac0 0x7f486007c3f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f485c030380 tx=0x7f485c032670 comp rx=0 tx=0).stop 2024-09-19T06:10:09.840 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:09.191+0000 7f4868489640 1 -- 172.21.15.53:0/884753210 shutdown_connections 2024-09-19T06:10:09.841 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:09.191+0000 7f4868489640 1 --2- 172.21.15.53:0/884753210 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f484c039230 0x7f484c03b6f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:09.841 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:09.191+0000 7f4868489640 1 --2- 172.21.15.53:0/884753210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4860106ac0 0x7f486007c3f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:09.841 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:09.191+0000 7f4868489640 1 -- 172.21.15.53:0/884753210 >> 172.21.15.53:0/884753210 conn(0x7f4860102330 msgr2=0x7f4860102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:09.841 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:09.191+0000 7f4868489640 1 -- 172.21.15.53:0/884753210 shutdown_connections 2024-09-19T06:10:09.841 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:09.191+0000 7f4868489640 1 -- 172.21.15.53:0/884753210 wait complete. 2024-09-19T06:10:09.841 INFO:teuthology.orchestra.run.smithi053.stdout:Deploying mon service with default placement... 2024-09-19T06:10:10.398 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:10 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:10.398 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:10 smithi053 ceph-mon[27968]: Added host smithi053 2024-09-19T06:10:10.398 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:10 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:10:11.189 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-19T06:10:11.189 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.393+0000 7f743c017640 1 Processor -- start 2024-09-19T06:10:11.189 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.393+0000 7f743c017640 1 -- start start 2024-09-19T06:10:11.189 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.393+0000 7f743c017640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74341059e0 0x7f7434105de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:11.189 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.393+0000 7f743c017640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f74341063b0 con 0x7f74341059e0 2024-09-19T06:10:11.190 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.394+0000 7f7439db3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74341059e0 0x7f7434105de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:11.190 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.394+0000 7f7439db3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74341059e0 0x7f7434105de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50334/0 (socket says 172.21.15.53:50334) 2024-09-19T06:10:11.190 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.394+0000 7f7439db3640 1 -- 172.21.15.53:0/1102622391 learned_addr learned my addr 172.21.15.53:0/1102622391 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:11.190 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.394+0000 7f7439db3640 1 -- 172.21.15.53:0/1102622391 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7434106be0 con 0x7f74341059e0 2024-09-19T06:10:11.190 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.395+0000 7f7439db3640 1 --2- 172.21.15.53:0/1102622391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74341059e0 0x7f7434105de0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7430009e80 tx=0x7f743002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecfed454e854a4ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:11.190 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.395+0000 7f7438db1640 1 -- 172.21.15.53:0/1102622391 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f7430036070 con 0x7f74341059e0 2024-09-19T06:10:11.190 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.395+0000 7f7438db1640 1 -- 172.21.15.53:0/1102622391 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7430031890 con 0x7f74341059e0 2024-09-19T06:10:11.190 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.395+0000 7f7438db1640 1 -- 172.21.15.53:0/1102622391 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f74300078c0 con 0x7f74341059e0 2024-09-19T06:10:11.190 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.395+0000 7f743c017640 1 -- 172.21.15.53:0/1102622391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74341059e0 msgr2=0x7f7434105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:11.190 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.395+0000 7f743c017640 1 --2- 172.21.15.53:0/1102622391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74341059e0 0x7f7434105de0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7430009e80 tx=0x7f743002a7a0 comp rx=0 tx=0).stop 2024-09-19T06:10:11.190 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.395+0000 7f743c017640 1 -- 172.21.15.53:0/1102622391 shutdown_connections 2024-09-19T06:10:11.190 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.395+0000 7f743c017640 1 --2- 172.21.15.53:0/1102622391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74341059e0 0x7f7434105de0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:11.191 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.395+0000 7f743c017640 1 -- 172.21.15.53:0/1102622391 >> 172.21.15.53:0/1102622391 conn(0x7f7434101150 msgr2=0x7f74341035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:11.191 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.396+0000 7f743c017640 1 -- 172.21.15.53:0/1102622391 shutdown_connections 2024-09-19T06:10:11.191 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.396+0000 7f743c017640 1 -- 172.21.15.53:0/1102622391 wait complete. 2024-09-19T06:10:11.191 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.396+0000 7f743c017640 1 Processor -- start 2024-09-19T06:10:11.191 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.396+0000 7f743c017640 1 -- start start 2024-09-19T06:10:11.191 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.396+0000 7f743c017640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74341059e0 0x7f743407a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:11.191 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.396+0000 7f743c017640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7434109f90 con 0x7f74341059e0 2024-09-19T06:10:11.191 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.396+0000 7f7439db3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74341059e0 0x7f743407a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:11.191 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.396+0000 7f7439db3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74341059e0 0x7f743407a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50342/0 (socket says 172.21.15.53:50342) 2024-09-19T06:10:11.191 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.396+0000 7f7439db3640 1 -- 172.21.15.53:0/1073317192 learned_addr learned my addr 172.21.15.53:0/1073317192 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:11.191 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.397+0000 7f7439db3640 1 -- 172.21.15.53:0/1073317192 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743419d7b0 con 0x7f74341059e0 2024-09-19T06:10:11.191 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.397+0000 7f7439db3640 1 --2- 172.21.15.53:0/1073317192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74341059e0 0x7f743407a310 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7430009680 tx=0x7f7430031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:11.191 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.397+0000 7f742affd640 1 -- 172.21.15.53:0/1073317192 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f7430045070 con 0x7f74341059e0 2024-09-19T06:10:11.192 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.397+0000 7f742affd640 1 -- 172.21.15.53:0/1073317192 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f743003a070 con 0x7f74341059e0 2024-09-19T06:10:11.192 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.397+0000 7f743c017640 1 -- 172.21.15.53:0/1073317192 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f743419d980 con 0x7f74341059e0 2024-09-19T06:10:11.192 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.397+0000 7f742affd640 1 -- 172.21.15.53:0/1073317192 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f743003f580 con 0x7f74341059e0 2024-09-19T06:10:11.192 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.397+0000 7f743c017640 1 -- 172.21.15.53:0/1073317192 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f743419de00 con 0x7f74341059e0 2024-09-19T06:10:11.192 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.398+0000 7f742affd640 1 -- 172.21.15.53:0/1073317192 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) ==== 46476+0+0 (secure 0 0 0) 0x7f7430048070 con 0x7f74341059e0 2024-09-19T06:10:11.192 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.398+0000 7f742affd640 1 --2- 172.21.15.53:0/1073317192 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f74200391e0 0x7f742003b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:11.192 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.398+0000 7f742affd640 1 -- 172.21.15.53:0/1073317192 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f743006c7d0 con 0x7f74341059e0 2024-09-19T06:10:11.192 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.398+0000 7f743c017640 1 -- 172.21.15.53:0/1073317192 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7434105e60 con 0x7f74341059e0 2024-09-19T06:10:11.192 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.399+0000 7f74395b2640 1 --2- 172.21.15.53:0/1073317192 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f74200391e0 0x7f742003b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:11.192 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.399+0000 7f74395b2640 1 --2- 172.21.15.53:0/1073317192 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f74200391e0 0x7f742003b6a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f74240098c0 tx=0x7f7424006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:11.192 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.403+0000 7f742affd640 1 -- 172.21.15.53:0/1073317192 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7430045da0 con 0x7f74341059e0 2024-09-19T06:10:11.192 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.599+0000 7f743c017640 1 -- 172.21.15.53:0/1073317192 --> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) -- 0x7f743419e030 con 0x7f74200391e0 2024-09-19T06:10:11.192 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.603+0000 7f742affd640 1 -- 172.21.15.53:0/1073317192 <== mgr.14118 v2:172.21.15.53:6800/681468213 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f743419e030 con 0x7f74200391e0 2024-09-19T06:10:11.193 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.606+0000 7f743c017640 1 -- 172.21.15.53:0/1073317192 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f74200391e0 msgr2=0x7f742003b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:11.193 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.606+0000 7f743c017640 1 --2- 172.21.15.53:0/1073317192 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f74200391e0 0x7f742003b6a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f74240098c0 tx=0x7f7424006db0 comp rx=0 tx=0).stop 2024-09-19T06:10:11.193 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.606+0000 7f743c017640 1 -- 172.21.15.53:0/1073317192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74341059e0 msgr2=0x7f743407a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:11.193 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.606+0000 7f743c017640 1 --2- 172.21.15.53:0/1073317192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74341059e0 0x7f743407a310 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7430009680 tx=0x7f7430031630 comp rx=0 tx=0).stop 2024-09-19T06:10:11.193 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.606+0000 7f743c017640 1 -- 172.21.15.53:0/1073317192 shutdown_connections 2024-09-19T06:10:11.193 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.606+0000 7f743c017640 1 --2- 172.21.15.53:0/1073317192 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f74200391e0 0x7f742003b6a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:11.193 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.606+0000 7f743c017640 1 --2- 172.21.15.53:0/1073317192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74341059e0 0x7f743407a310 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:11.193 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.606+0000 7f743c017640 1 -- 172.21.15.53:0/1073317192 >> 172.21.15.53:0/1073317192 conn(0x7f7434101150 msgr2=0x7f7434101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:11.193 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.606+0000 7f743c017640 1 -- 172.21.15.53:0/1073317192 shutdown_connections 2024-09-19T06:10:11.193 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:10.606+0000 7f743c017640 1 -- 172.21.15.53:0/1073317192 wait complete. 2024-09-19T06:10:11.193 INFO:teuthology.orchestra.run.smithi053.stdout:Deploying mgr service with default placement... 2024-09-19T06:10:11.851 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:11 smithi053 ceph-mon[27968]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:10:11.852 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:11 smithi053 ceph-mon[27968]: Saving service mon spec with placement count:5 2024-09-19T06:10:11.852 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:11 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:13.181 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:12 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:13.181 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:12 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:13.181 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:12 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:13.894 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-19T06:10:13.895 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.375+0000 7fe7b6ccd640 1 Processor -- start 2024-09-19T06:10:13.895 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.375+0000 7fe7b6ccd640 1 -- start start 2024-09-19T06:10:13.895 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.375+0000 7fe7b6ccd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7b0107550 0x7fe7b0109940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:13.895 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.375+0000 7fe7b6ccd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b007a730 con 0x7fe7b0107550 2024-09-19T06:10:13.895 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.375+0000 7fe7b4a69640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7b0107550 0x7fe7b0109940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:13.895 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.375+0000 7fe7b4a69640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7b0107550 0x7fe7b0109940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47844/0 (socket says 172.21.15.53:47844) 2024-09-19T06:10:13.895 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.375+0000 7fe7b4a69640 1 -- 172.21.15.53:0/82601730 learned_addr learned my addr 172.21.15.53:0/82601730 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:13.895 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.375+0000 7fe7b4a69640 1 -- 172.21.15.53:0/82601730 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b0109e80 con 0x7fe7b0107550 2024-09-19T06:10:13.895 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.376+0000 7fe7b4a69640 1 --2- 172.21.15.53:0/82601730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7b0107550 0x7fe7b0109940 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4009820 tx=0x7fe7a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de15cedfdb2de10b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:13.895 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.376+0000 7fe7af7fe640 1 -- 172.21.15.53:0/82601730 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fe7a4037070 con 0x7fe7b0107550 2024-09-19T06:10:13.895 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.376+0000 7fe7af7fe640 1 -- 172.21.15.53:0/82601730 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7a402ad40 con 0x7fe7b0107550 2024-09-19T06:10:13.895 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.376+0000 7fe7af7fe640 1 -- 172.21.15.53:0/82601730 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fe7a4030620 con 0x7fe7b0107550 2024-09-19T06:10:13.895 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.377+0000 7fe7b6ccd640 1 -- 172.21.15.53:0/82601730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7b0107550 msgr2=0x7fe7b0109940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:13.896 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.377+0000 7fe7b6ccd640 1 --2- 172.21.15.53:0/82601730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7b0107550 0x7fe7b0109940 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4009820 tx=0x7fe7a402a270 comp rx=0 tx=0).stop 2024-09-19T06:10:13.896 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.377+0000 7fe7b6ccd640 1 -- 172.21.15.53:0/82601730 shutdown_connections 2024-09-19T06:10:13.896 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.377+0000 7fe7b6ccd640 1 --2- 172.21.15.53:0/82601730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7b0107550 0x7fe7b0109940 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:13.896 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.377+0000 7fe7b6ccd640 1 -- 172.21.15.53:0/82601730 >> 172.21.15.53:0/82601730 conn(0x7fe7b0101150 msgr2=0x7fe7b01035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:13.896 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.377+0000 7fe7b6ccd640 1 -- 172.21.15.53:0/82601730 shutdown_connections 2024-09-19T06:10:13.896 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.377+0000 7fe7b6ccd640 1 -- 172.21.15.53:0/82601730 wait complete. 2024-09-19T06:10:13.896 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.378+0000 7fe7b6ccd640 1 Processor -- start 2024-09-19T06:10:13.896 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.378+0000 7fe7b6ccd640 1 -- start start 2024-09-19T06:10:13.896 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.378+0000 7fe7b6ccd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7b0107550 0x7fe7b019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:13.896 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.378+0000 7fe7b6ccd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b010af00 con 0x7fe7b0107550 2024-09-19T06:10:13.896 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.378+0000 7fe7b4a69640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7b0107550 0x7fe7b019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:13.896 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.378+0000 7fe7b4a69640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7b0107550 0x7fe7b019cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47858/0 (socket says 172.21.15.53:47858) 2024-09-19T06:10:13.896 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.378+0000 7fe7b4a69640 1 -- 172.21.15.53:0/2645791410 learned_addr learned my addr 172.21.15.53:0/2645791410 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:13.897 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.378+0000 7fe7b4a69640 1 -- 172.21.15.53:0/2645791410 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b019d1d0 con 0x7fe7b0107550 2024-09-19T06:10:13.897 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.378+0000 7fe7b4a69640 1 --2- 172.21.15.53:0/2645791410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7b0107550 0x7fe7b019cc90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4006f10 tx=0x7fe7a4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:13.897 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.378+0000 7fe7adffb640 1 -- 172.21.15.53:0/2645791410 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fe7a4032a70 con 0x7fe7b0107550 2024-09-19T06:10:13.897 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.378+0000 7fe7b6ccd640 1 -- 172.21.15.53:0/2645791410 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b019d460 con 0x7fe7b0107550 2024-09-19T06:10:13.897 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.379+0000 7fe7adffb640 1 -- 172.21.15.53:0/2645791410 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7a4032c10 con 0x7fe7b0107550 2024-09-19T06:10:13.897 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.379+0000 7fe7adffb640 1 -- 172.21.15.53:0/2645791410 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fe7a403d620 con 0x7fe7b0107550 2024-09-19T06:10:13.897 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.379+0000 7fe7b6ccd640 1 -- 172.21.15.53:0/2645791410 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b01a0150 con 0x7fe7b0107550 2024-09-19T06:10:13.897 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.380+0000 7fe7adffb640 1 -- 172.21.15.53:0/2645791410 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) ==== 46476+0+0 (secure 0 0 0) 0x7fe7a403d920 con 0x7fe7b0107550 2024-09-19T06:10:13.897 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.380+0000 7fe7b6ccd640 1 -- 172.21.15.53:0/2645791410 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7b01079b0 con 0x7fe7b0107550 2024-09-19T06:10:13.897 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.380+0000 7fe7adffb640 1 --2- 172.21.15.53:0/2645791410 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fe7a00391e0 0x7fe7a003b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:13.897 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.380+0000 7fe7adffb640 1 -- 172.21.15.53:0/2645791410 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe7a406bff0 con 0x7fe7b0107550 2024-09-19T06:10:13.897 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.380+0000 7fe7affff640 1 --2- 172.21.15.53:0/2645791410 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fe7a00391e0 0x7fe7a003b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:13.897 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.381+0000 7fe7affff640 1 --2- 172.21.15.53:0/2645791410 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fe7a00391e0 0x7fe7a003b6a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe79c0098c0 tx=0x7fe79c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:13.898 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.385+0000 7fe7adffb640 1 -- 172.21.15.53:0/2645791410 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe7a40454d0 con 0x7fe7b0107550 2024-09-19T06:10:13.898 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.583+0000 7fe7b6ccd640 1 -- 172.21.15.53:0/2645791410 --> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) -- 0x7fe7b0102370 con 0x7fe7a00391e0 2024-09-19T06:10:13.898 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.588+0000 7fe7adffb640 1 -- 172.21.15.53:0/2645791410 <== mgr.14118 v2:172.21.15.53:6800/681468213 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe7b0102370 con 0x7fe7a00391e0 2024-09-19T06:10:13.898 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.591+0000 7fe7b6ccd640 1 -- 172.21.15.53:0/2645791410 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fe7a00391e0 msgr2=0x7fe7a003b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:13.898 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.591+0000 7fe7b6ccd640 1 --2- 172.21.15.53:0/2645791410 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fe7a00391e0 0x7fe7a003b6a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe79c0098c0 tx=0x7fe79c006db0 comp rx=0 tx=0).stop 2024-09-19T06:10:13.898 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.591+0000 7fe7b6ccd640 1 -- 172.21.15.53:0/2645791410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7b0107550 msgr2=0x7fe7b019cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:13.898 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.591+0000 7fe7b6ccd640 1 --2- 172.21.15.53:0/2645791410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7b0107550 0x7fe7b019cc90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4006f10 tx=0x7fe7a4032670 comp rx=0 tx=0).stop 2024-09-19T06:10:13.898 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.591+0000 7fe7b6ccd640 1 -- 172.21.15.53:0/2645791410 shutdown_connections 2024-09-19T06:10:13.898 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.591+0000 7fe7b6ccd640 1 --2- 172.21.15.53:0/2645791410 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fe7a00391e0 0x7fe7a003b6a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:13.898 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.591+0000 7fe7b6ccd640 1 --2- 172.21.15.53:0/2645791410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7b0107550 0x7fe7b019cc90 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:13.898 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.591+0000 7fe7b6ccd640 1 -- 172.21.15.53:0/2645791410 >> 172.21.15.53:0/2645791410 conn(0x7fe7b0101150 msgr2=0x7fe7b0101c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:13.898 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.591+0000 7fe7b6ccd640 1 -- 172.21.15.53:0/2645791410 shutdown_connections 2024-09-19T06:10:13.898 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:12.591+0000 7fe7b6ccd640 1 -- 172.21.15.53:0/2645791410 wait complete. 2024-09-19T06:10:13.899 INFO:teuthology.orchestra.run.smithi053.stdout:Deploying crash service with default placement... 2024-09-19T06:10:14.214 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:13 smithi053 ceph-mon[27968]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:10:14.215 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:13 smithi053 ceph-mon[27968]: Saving service mgr spec with placement count:2 2024-09-19T06:10:14.215 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:13 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:15.084 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-09-19T06:10:15.084 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.296+0000 7f84665b4640 1 Processor -- start 2024-09-19T06:10:15.084 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.296+0000 7f84665b4640 1 -- start start 2024-09-19T06:10:15.084 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.296+0000 7f84665b4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8460107630 0x7f8460109a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:15.084 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.296+0000 7f84665b4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f846007a650 con 0x7f8460107630 2024-09-19T06:10:15.084 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.296+0000 7f845ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8460107630 0x7f8460109a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:15.084 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.296+0000 7f845ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8460107630 0x7f8460109a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47874/0 (socket says 172.21.15.53:47874) 2024-09-19T06:10:15.084 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.296+0000 7f845ffff640 1 -- 172.21.15.53:0/2140284853 learned_addr learned my addr 172.21.15.53:0/2140284853 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:15.085 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.296+0000 7f845ffff640 1 -- 172.21.15.53:0/2140284853 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8460109f60 con 0x7f8460107630 2024-09-19T06:10:15.085 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.297+0000 7f845ffff640 1 --2- 172.21.15.53:0/2140284853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8460107630 0x7f8460109a20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8450009820 tx=0x7f845002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7517d6998bb65539 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:15.085 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.298+0000 7f845effd640 1 -- 172.21.15.53:0/2140284853 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f8450037070 con 0x7f8460107630 2024-09-19T06:10:15.085 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.298+0000 7f845effd640 1 -- 172.21.15.53:0/2140284853 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f845002ad40 con 0x7f8460107630 2024-09-19T06:10:15.085 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.298+0000 7f845effd640 1 -- 172.21.15.53:0/2140284853 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f8450030620 con 0x7f8460107630 2024-09-19T06:10:15.085 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.298+0000 7f84665b4640 1 -- 172.21.15.53:0/2140284853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8460107630 msgr2=0x7f8460109a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:15.085 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.298+0000 7f84665b4640 1 --2- 172.21.15.53:0/2140284853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8460107630 0x7f8460109a20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8450009820 tx=0x7f845002a270 comp rx=0 tx=0).stop 2024-09-19T06:10:15.085 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.298+0000 7f84665b4640 1 -- 172.21.15.53:0/2140284853 shutdown_connections 2024-09-19T06:10:15.085 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.298+0000 7f84665b4640 1 --2- 172.21.15.53:0/2140284853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8460107630 0x7f8460109a20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:15.085 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.299+0000 7f84665b4640 1 -- 172.21.15.53:0/2140284853 >> 172.21.15.53:0/2140284853 conn(0x7f8460101150 msgr2=0x7f84601035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:15.085 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.299+0000 7f84665b4640 1 -- 172.21.15.53:0/2140284853 shutdown_connections 2024-09-19T06:10:15.085 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.299+0000 7f84665b4640 1 -- 172.21.15.53:0/2140284853 wait complete. 2024-09-19T06:10:15.085 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.300+0000 7f84665b4640 1 Processor -- start 2024-09-19T06:10:15.085 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.300+0000 7f84665b4640 1 -- start start 2024-09-19T06:10:15.086 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.300+0000 7f84665b4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8460107630 0x7f846019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:15.086 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.300+0000 7f84665b4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f846010aec0 con 0x7f8460107630 2024-09-19T06:10:15.086 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.300+0000 7f845ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8460107630 0x7f846019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:15.086 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.300+0000 7f845ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8460107630 0x7f846019aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47876/0 (socket says 172.21.15.53:47876) 2024-09-19T06:10:15.086 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.300+0000 7f845ffff640 1 -- 172.21.15.53:0/873129047 learned_addr learned my addr 172.21.15.53:0/873129047 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:15.086 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.300+0000 7f845ffff640 1 -- 172.21.15.53:0/873129047 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846019afe0 con 0x7f8460107630 2024-09-19T06:10:15.086 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.301+0000 7f845ffff640 1 --2- 172.21.15.53:0/873129047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8460107630 0x7f846019aaa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8450000c00 tx=0x7f8450030fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:15.086 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.301+0000 7f845d7fa640 1 -- 172.21.15.53:0/873129047 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f8450032a00 con 0x7f8460107630 2024-09-19T06:10:15.086 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.301+0000 7f845d7fa640 1 -- 172.21.15.53:0/873129047 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8450032ba0 con 0x7f8460107630 2024-09-19T06:10:15.086 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.301+0000 7f845d7fa640 1 -- 172.21.15.53:0/873129047 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f845003d5d0 con 0x7f8460107630 2024-09-19T06:10:15.086 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.301+0000 7f84665b4640 1 -- 172.21.15.53:0/873129047 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f846019b270 con 0x7f8460107630 2024-09-19T06:10:15.086 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.301+0000 7f84665b4640 1 -- 172.21.15.53:0/873129047 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f846019b690 con 0x7f8460107630 2024-09-19T06:10:15.086 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.302+0000 7f845d7fa640 1 -- 172.21.15.53:0/873129047 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) ==== 46476+0+0 (secure 0 0 0) 0x7f845003d880 con 0x7f8460107630 2024-09-19T06:10:15.086 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.302+0000 7f845d7fa640 1 --2- 172.21.15.53:0/873129047 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f8448039290 0x7f844803b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:15.087 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.302+0000 7f845d7fa640 1 -- 172.21.15.53:0/873129047 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f845006be00 con 0x7f8460107630 2024-09-19T06:10:15.087 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.302+0000 7f84665b4640 1 -- 172.21.15.53:0/873129047 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8460107a90 con 0x7f8460107630 2024-09-19T06:10:15.087 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.303+0000 7f845f7fe640 1 --2- 172.21.15.53:0/873129047 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f8448039290 0x7f844803b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:15.087 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.303+0000 7f845f7fe640 1 --2- 172.21.15.53:0/873129047 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f8448039290 0x7f844803b750 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f84540098c0 tx=0x7f8454006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:15.087 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.308+0000 7f845d7fa640 1 -- 172.21.15.53:0/873129047 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f84500454d0 con 0x7f8460107630 2024-09-19T06:10:15.087 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.498+0000 7f84665b4640 1 -- 172.21.15.53:0/873129047 --> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) -- 0x7f8460102440 con 0x7f8448039290 2024-09-19T06:10:15.087 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.502+0000 7f845d7fa640 1 -- 172.21.15.53:0/873129047 <== mgr.14118 v2:172.21.15.53:6800/681468213 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f8460102440 con 0x7f8448039290 2024-09-19T06:10:15.087 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.505+0000 7f84665b4640 1 -- 172.21.15.53:0/873129047 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f8448039290 msgr2=0x7f844803b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:15.087 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.505+0000 7f84665b4640 1 --2- 172.21.15.53:0/873129047 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f8448039290 0x7f844803b750 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f84540098c0 tx=0x7f8454006db0 comp rx=0 tx=0).stop 2024-09-19T06:10:15.087 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.505+0000 7f84665b4640 1 -- 172.21.15.53:0/873129047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8460107630 msgr2=0x7f846019aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:15.087 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.505+0000 7f84665b4640 1 --2- 172.21.15.53:0/873129047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8460107630 0x7f846019aaa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8450000c00 tx=0x7f8450030fc0 comp rx=0 tx=0).stop 2024-09-19T06:10:15.087 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.505+0000 7f84665b4640 1 -- 172.21.15.53:0/873129047 shutdown_connections 2024-09-19T06:10:15.087 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.505+0000 7f84665b4640 1 --2- 172.21.15.53:0/873129047 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f8448039290 0x7f844803b750 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:15.087 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.505+0000 7f84665b4640 1 --2- 172.21.15.53:0/873129047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8460107630 0x7f846019aaa0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:15.088 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.505+0000 7f84665b4640 1 -- 172.21.15.53:0/873129047 >> 172.21.15.53:0/873129047 conn(0x7f8460101150 msgr2=0x7f8460101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:15.088 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.505+0000 7f84665b4640 1 -- 172.21.15.53:0/873129047 shutdown_connections 2024-09-19T06:10:15.088 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:14.505+0000 7f84665b4640 1 -- 172.21.15.53:0/873129047 wait complete. 2024-09-19T06:10:15.088 INFO:teuthology.orchestra.run.smithi053.stdout:Deploying ceph-exporter service with default placement... 2024-09-19T06:10:15.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:15 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:15.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:15 smithi053 ceph-mon[27968]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:10:15.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:15 smithi053 ceph-mon[27968]: Saving service crash spec with placement * 2024-09-19T06:10:15.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:15 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:16.317 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-09-19T06:10:16.317 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.458+0000 7f4c26423640 1 Processor -- start 2024-09-19T06:10:16.317 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.458+0000 7f4c26423640 1 -- start start 2024-09-19T06:10:16.318 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.458+0000 7f4c26423640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c20106af0 0x7f4c20106ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:16.318 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.458+0000 7f4c26423640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4c20107430 con 0x7f4c20106af0 2024-09-19T06:10:16.318 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.458+0000 7f4c1ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c20106af0 0x7f4c20106ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:16.318 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.459+0000 7f4c1ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c20106af0 0x7f4c20106ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47886/0 (socket says 172.21.15.53:47886) 2024-09-19T06:10:16.318 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.459+0000 7f4c1ffff640 1 -- 172.21.15.53:0/3096586442 learned_addr learned my addr 172.21.15.53:0/3096586442 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:16.318 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.459+0000 7f4c1ffff640 1 -- 172.21.15.53:0/3096586442 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c201075b0 con 0x7f4c20106af0 2024-09-19T06:10:16.318 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.459+0000 7f4c1ffff640 1 --2- 172.21.15.53:0/3096586442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c20106af0 0x7f4c20106ef0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4c10009820 tx=0x7f4c1002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abbabfe3b42f9f9a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:16.318 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.460+0000 7f4c1effd640 1 -- 172.21.15.53:0/3096586442 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f4c10037070 con 0x7f4c20106af0 2024-09-19T06:10:16.318 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.460+0000 7f4c1effd640 1 -- 172.21.15.53:0/3096586442 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c1002ad40 con 0x7f4c20106af0 2024-09-19T06:10:16.318 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.460+0000 7f4c1effd640 1 -- 172.21.15.53:0/3096586442 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f4c10030620 con 0x7f4c20106af0 2024-09-19T06:10:16.318 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.460+0000 7f4c26423640 1 -- 172.21.15.53:0/3096586442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c20106af0 msgr2=0x7f4c20106ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:16.318 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.460+0000 7f4c26423640 1 --2- 172.21.15.53:0/3096586442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c20106af0 0x7f4c20106ef0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4c10009820 tx=0x7f4c1002a270 comp rx=0 tx=0).stop 2024-09-19T06:10:16.318 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.460+0000 7f4c26423640 1 -- 172.21.15.53:0/3096586442 shutdown_connections 2024-09-19T06:10:16.318 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.460+0000 7f4c26423640 1 --2- 172.21.15.53:0/3096586442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c20106af0 0x7f4c20106ef0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:16.319 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.460+0000 7f4c26423640 1 -- 172.21.15.53:0/3096586442 >> 172.21.15.53:0/3096586442 conn(0x7f4c201023c0 msgr2=0x7f4c201047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:16.319 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.460+0000 7f4c26423640 1 -- 172.21.15.53:0/3096586442 shutdown_connections 2024-09-19T06:10:16.319 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.460+0000 7f4c26423640 1 -- 172.21.15.53:0/3096586442 wait complete. 2024-09-19T06:10:16.319 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.461+0000 7f4c26423640 1 Processor -- start 2024-09-19T06:10:16.319 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.461+0000 7f4c26423640 1 -- start start 2024-09-19T06:10:16.319 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.461+0000 7f4c26423640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c20106af0 0x7f4c2019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:16.319 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.461+0000 7f4c26423640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2010af60 con 0x7f4c20106af0 2024-09-19T06:10:16.319 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.461+0000 7f4c1ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c20106af0 0x7f4c2019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:16.319 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.461+0000 7f4c1ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c20106af0 0x7f4c2019aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47888/0 (socket says 172.21.15.53:47888) 2024-09-19T06:10:16.319 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.461+0000 7f4c1ffff640 1 -- 172.21.15.53:0/2554990760 learned_addr learned my addr 172.21.15.53:0/2554990760 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:16.319 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.462+0000 7f4c1ffff640 1 -- 172.21.15.53:0/2554990760 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c2019af50 con 0x7f4c20106af0 2024-09-19T06:10:16.319 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.462+0000 7f4c1ffff640 1 --2- 172.21.15.53:0/2554990760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c20106af0 0x7f4c2019aa10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4c10000c00 tx=0x7f4c10030fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:16.319 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.462+0000 7f4c1d7fa640 1 -- 172.21.15.53:0/2554990760 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f4c10032a00 con 0x7f4c20106af0 2024-09-19T06:10:16.319 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.462+0000 7f4c1d7fa640 1 -- 172.21.15.53:0/2554990760 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c10032ba0 con 0x7f4c20106af0 2024-09-19T06:10:16.320 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.462+0000 7f4c26423640 1 -- 172.21.15.53:0/2554990760 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c2019b1e0 con 0x7f4c20106af0 2024-09-19T06:10:16.320 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.462+0000 7f4c1d7fa640 1 -- 172.21.15.53:0/2554990760 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f4c1003d5d0 con 0x7f4c20106af0 2024-09-19T06:10:16.320 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.462+0000 7f4c26423640 1 -- 172.21.15.53:0/2554990760 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c2019ded0 con 0x7f4c20106af0 2024-09-19T06:10:16.320 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.463+0000 7f4c1d7fa640 1 -- 172.21.15.53:0/2554990760 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) ==== 46476+0+0 (secure 0 0 0) 0x7f4c1003d880 con 0x7f4c20106af0 2024-09-19T06:10:16.320 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.463+0000 7f4c1d7fa640 1 --2- 172.21.15.53:0/2554990760 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f4c08039230 0x7f4c0803b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:16.320 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.463+0000 7f4c1d7fa640 1 -- 172.21.15.53:0/2554990760 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4c1006bdd0 con 0x7f4c20106af0 2024-09-19T06:10:16.320 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.463+0000 7f4c26423640 1 -- 172.21.15.53:0/2554990760 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c20106ef0 con 0x7f4c20106af0 2024-09-19T06:10:16.320 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.463+0000 7f4c1f7fe640 1 --2- 172.21.15.53:0/2554990760 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f4c08039230 0x7f4c0803b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:16.320 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.464+0000 7f4c1f7fe640 1 --2- 172.21.15.53:0/2554990760 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f4c08039230 0x7f4c0803b6f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4c140098c0 tx=0x7f4c14006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:16.320 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.468+0000 7f4c1d7fa640 1 -- 172.21.15.53:0/2554990760 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4c100454d0 con 0x7f4c20106af0 2024-09-19T06:10:16.320 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.660+0000 7f4c26423640 1 -- 172.21.15.53:0/2554990760 --> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}) -- 0x7f4c201035a0 con 0x7f4c08039230 2024-09-19T06:10:16.320 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.664+0000 7f4c1d7fa640 1 -- 172.21.15.53:0/2554990760 <== mgr.14118 v2:172.21.15.53:6800/681468213 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+34 (secure 0 0 0) 0x7f4c201035a0 con 0x7f4c08039230 2024-09-19T06:10:16.320 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.666+0000 7f4c26423640 1 -- 172.21.15.53:0/2554990760 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f4c08039230 msgr2=0x7f4c0803b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:16.321 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.666+0000 7f4c26423640 1 --2- 172.21.15.53:0/2554990760 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f4c08039230 0x7f4c0803b6f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4c140098c0 tx=0x7f4c14006db0 comp rx=0 tx=0).stop 2024-09-19T06:10:16.321 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.666+0000 7f4c26423640 1 -- 172.21.15.53:0/2554990760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c20106af0 msgr2=0x7f4c2019aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:16.321 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.666+0000 7f4c26423640 1 --2- 172.21.15.53:0/2554990760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c20106af0 0x7f4c2019aa10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4c10000c00 tx=0x7f4c10030fc0 comp rx=0 tx=0).stop 2024-09-19T06:10:16.321 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.667+0000 7f4c26423640 1 -- 172.21.15.53:0/2554990760 shutdown_connections 2024-09-19T06:10:16.321 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.667+0000 7f4c26423640 1 --2- 172.21.15.53:0/2554990760 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f4c08039230 0x7f4c0803b6f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:16.321 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.667+0000 7f4c26423640 1 --2- 172.21.15.53:0/2554990760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c20106af0 0x7f4c2019aa10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:16.321 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.667+0000 7f4c26423640 1 -- 172.21.15.53:0/2554990760 >> 172.21.15.53:0/2554990760 conn(0x7f4c201023c0 msgr2=0x7f4c20102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:16.321 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.667+0000 7f4c26423640 1 -- 172.21.15.53:0/2554990760 shutdown_connections 2024-09-19T06:10:16.321 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:15.667+0000 7f4c26423640 1 -- 172.21.15.53:0/2554990760 wait complete. 2024-09-19T06:10:16.321 INFO:teuthology.orchestra.run.smithi053.stdout:Deploying prometheus service with default placement... 2024-09-19T06:10:16.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:16 smithi053 ceph-mon[27968]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:10:16.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:16 smithi053 ceph-mon[27968]: Saving service ceph-exporter spec with placement * 2024-09-19T06:10:16.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:16 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:17.936 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:17 smithi053 ceph-mon[27968]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:10:17.937 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:17 smithi053 ceph-mon[27968]: Saving service prometheus spec with placement count:1 2024-09-19T06:10:17.937 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:17 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:17.945 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-09-19T06:10:17.945 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.658+0000 7f1d3c3a3640 1 Processor -- start 2024-09-19T06:10:17.945 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.658+0000 7f1d3c3a3640 1 -- start start 2024-09-19T06:10:17.945 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.658+0000 7f1d3c3a3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d34106af0 0x7f1d34106ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:17.945 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.658+0000 7f1d3c3a3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1d34107430 con 0x7f1d34106af0 2024-09-19T06:10:17.945 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.658+0000 7f1d3a13f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d34106af0 0x7f1d34106ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:17.945 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.658+0000 7f1d3a13f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d34106af0 0x7f1d34106ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47890/0 (socket says 172.21.15.53:47890) 2024-09-19T06:10:17.946 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.658+0000 7f1d3a13f640 1 -- 172.21.15.53:0/805082382 learned_addr learned my addr 172.21.15.53:0/805082382 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:17.946 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.659+0000 7f1d3a13f640 1 -- 172.21.15.53:0/805082382 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d341075b0 con 0x7f1d34106af0 2024-09-19T06:10:17.946 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.659+0000 7f1d3a13f640 1 --2- 172.21.15.53:0/805082382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d34106af0 0x7f1d34106ef0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009820 tx=0x7f1d3002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fade6e4056dc8d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:17.946 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.659+0000 7f1d3913d640 1 -- 172.21.15.53:0/805082382 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f1d30037070 con 0x7f1d34106af0 2024-09-19T06:10:17.946 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.659+0000 7f1d3913d640 1 -- 172.21.15.53:0/805082382 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d3002ad40 con 0x7f1d34106af0 2024-09-19T06:10:17.946 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.659+0000 7f1d3913d640 1 -- 172.21.15.53:0/805082382 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f1d30030620 con 0x7f1d34106af0 2024-09-19T06:10:17.946 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.660+0000 7f1d3c3a3640 1 -- 172.21.15.53:0/805082382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d34106af0 msgr2=0x7f1d34106ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:17.946 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.660+0000 7f1d3c3a3640 1 --2- 172.21.15.53:0/805082382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d34106af0 0x7f1d34106ef0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009820 tx=0x7f1d3002a270 comp rx=0 tx=0).stop 2024-09-19T06:10:17.947 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.660+0000 7f1d3c3a3640 1 -- 172.21.15.53:0/805082382 shutdown_connections 2024-09-19T06:10:17.947 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.660+0000 7f1d3c3a3640 1 --2- 172.21.15.53:0/805082382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d34106af0 0x7f1d34106ef0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:17.947 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.660+0000 7f1d3c3a3640 1 -- 172.21.15.53:0/805082382 >> 172.21.15.53:0/805082382 conn(0x7f1d341023c0 msgr2=0x7f1d341047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:17.947 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.660+0000 7f1d3c3a3640 1 -- 172.21.15.53:0/805082382 shutdown_connections 2024-09-19T06:10:17.947 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.660+0000 7f1d3c3a3640 1 -- 172.21.15.53:0/805082382 wait complete. 2024-09-19T06:10:17.947 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.661+0000 7f1d3c3a3640 1 Processor -- start 2024-09-19T06:10:17.947 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.661+0000 7f1d3c3a3640 1 -- start start 2024-09-19T06:10:17.947 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.661+0000 7f1d3c3a3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d34106af0 0x7f1d3419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:17.947 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.661+0000 7f1d3c3a3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3410af60 con 0x7f1d34106af0 2024-09-19T06:10:17.947 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.661+0000 7f1d3a13f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d34106af0 0x7f1d3419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:17.947 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.661+0000 7f1d3a13f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d34106af0 0x7f1d3419aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47904/0 (socket says 172.21.15.53:47904) 2024-09-19T06:10:17.947 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.661+0000 7f1d3a13f640 1 -- 172.21.15.53:0/2337754143 learned_addr learned my addr 172.21.15.53:0/2337754143 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:17.947 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.661+0000 7f1d3a13f640 1 -- 172.21.15.53:0/2337754143 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3419af50 con 0x7f1d34106af0 2024-09-19T06:10:17.948 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.662+0000 7f1d3a13f640 1 --2- 172.21.15.53:0/2337754143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d34106af0 0x7f1d3419aa10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1d30000c00 tx=0x7f1d30030fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:17.948 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.662+0000 7f1d2b7fe640 1 -- 172.21.15.53:0/2337754143 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f1d30032a00 con 0x7f1d34106af0 2024-09-19T06:10:17.948 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.662+0000 7f1d2b7fe640 1 -- 172.21.15.53:0/2337754143 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d30032ba0 con 0x7f1d34106af0 2024-09-19T06:10:17.948 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.662+0000 7f1d3c3a3640 1 -- 172.21.15.53:0/2337754143 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d3419b1e0 con 0x7f1d34106af0 2024-09-19T06:10:17.948 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.662+0000 7f1d2b7fe640 1 -- 172.21.15.53:0/2337754143 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f1d3003d5d0 con 0x7f1d34106af0 2024-09-19T06:10:17.948 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.662+0000 7f1d3c3a3640 1 -- 172.21.15.53:0/2337754143 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d3419ded0 con 0x7f1d34106af0 2024-09-19T06:10:17.948 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.663+0000 7f1d2b7fe640 1 -- 172.21.15.53:0/2337754143 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) ==== 46476+0+0 (secure 0 0 0) 0x7f1d3003e430 con 0x7f1d34106af0 2024-09-19T06:10:17.948 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.663+0000 7f1d3c3a3640 1 -- 172.21.15.53:0/2337754143 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d34106ef0 con 0x7f1d34106af0 2024-09-19T06:10:17.948 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.663+0000 7f1d2b7fe640 1 --2- 172.21.15.53:0/2337754143 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f1d20039230 0x7f1d2003b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:17.948 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.663+0000 7f1d2b7fe640 1 -- 172.21.15.53:0/2337754143 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1d3006be00 con 0x7f1d34106af0 2024-09-19T06:10:17.948 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.663+0000 7f1d3993e640 1 --2- 172.21.15.53:0/2337754143 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f1d20039230 0x7f1d2003b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:17.948 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.664+0000 7f1d3993e640 1 --2- 172.21.15.53:0/2337754143 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f1d20039230 0x7f1d2003b6f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1d240098c0 tx=0x7f1d24006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:17.948 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.670+0000 7f1d2b7fe640 1 -- 172.21.15.53:0/2337754143 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1d3003e720 con 0x7f1d34106af0 2024-09-19T06:10:17.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.862+0000 7f1d3c3a3640 1 -- 172.21.15.53:0/2337754143 --> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) -- 0x7f1d341035a0 con 0x7f1d20039230 2024-09-19T06:10:17.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.866+0000 7f1d2b7fe640 1 -- 172.21.15.53:0/2337754143 <== mgr.14118 v2:172.21.15.53:6800/681468213 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+31 (secure 0 0 0) 0x7f1d341035a0 con 0x7f1d20039230 2024-09-19T06:10:17.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.868+0000 7f1d3c3a3640 1 -- 172.21.15.53:0/2337754143 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f1d20039230 msgr2=0x7f1d2003b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:17.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.868+0000 7f1d3c3a3640 1 --2- 172.21.15.53:0/2337754143 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f1d20039230 0x7f1d2003b6f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1d240098c0 tx=0x7f1d24006db0 comp rx=0 tx=0).stop 2024-09-19T06:10:17.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.868+0000 7f1d3c3a3640 1 -- 172.21.15.53:0/2337754143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d34106af0 msgr2=0x7f1d3419aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:17.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.868+0000 7f1d3c3a3640 1 --2- 172.21.15.53:0/2337754143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d34106af0 0x7f1d3419aa10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1d30000c00 tx=0x7f1d30030fc0 comp rx=0 tx=0).stop 2024-09-19T06:10:17.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.868+0000 7f1d3c3a3640 1 -- 172.21.15.53:0/2337754143 shutdown_connections 2024-09-19T06:10:17.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.868+0000 7f1d3c3a3640 1 --2- 172.21.15.53:0/2337754143 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f1d20039230 0x7f1d2003b6f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:17.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.868+0000 7f1d3c3a3640 1 --2- 172.21.15.53:0/2337754143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d34106af0 0x7f1d3419aa10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:17.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.868+0000 7f1d3c3a3640 1 -- 172.21.15.53:0/2337754143 >> 172.21.15.53:0/2337754143 conn(0x7f1d341023c0 msgr2=0x7f1d34102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:17.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.869+0000 7f1d3c3a3640 1 -- 172.21.15.53:0/2337754143 shutdown_connections 2024-09-19T06:10:17.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:16.869+0000 7f1d3c3a3640 1 -- 172.21.15.53:0/2337754143 wait complete. 2024-09-19T06:10:17.949 INFO:teuthology.orchestra.run.smithi053.stdout:Deploying grafana service with default placement... 2024-09-19T06:10:19.118 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-09-19T06:10:19.118 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.365+0000 7fbd0ff1c640 1 Processor -- start 2024-09-19T06:10:19.118 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.366+0000 7fbd0ff1c640 1 -- start start 2024-09-19T06:10:19.118 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.366+0000 7fbd0ff1c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd08106af0 0x7fbd08106ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:19.118 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.366+0000 7fbd0ff1c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbd08107430 con 0x7fbd08106af0 2024-09-19T06:10:19.118 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.366+0000 7fbd0dcb8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd08106af0 0x7fbd08106ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:19.118 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.366+0000 7fbd0dcb8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd08106af0 0x7fbd08106ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47908/0 (socket says 172.21.15.53:47908) 2024-09-19T06:10:19.118 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.366+0000 7fbd0dcb8640 1 -- 172.21.15.53:0/3187933874 learned_addr learned my addr 172.21.15.53:0/3187933874 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:19.118 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.366+0000 7fbd0dcb8640 1 -- 172.21.15.53:0/3187933874 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd081075b0 con 0x7fbd08106af0 2024-09-19T06:10:19.118 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.367+0000 7fbd0dcb8640 1 --2- 172.21.15.53:0/3187933874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd08106af0 0x7fbd08106ef0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbd04009820 tx=0x7fbd0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f019cd349f09d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:19.119 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.367+0000 7fbd0ccb6640 1 -- 172.21.15.53:0/3187933874 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fbd04037070 con 0x7fbd08106af0 2024-09-19T06:10:19.119 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.367+0000 7fbd0ccb6640 1 -- 172.21.15.53:0/3187933874 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd0402ad40 con 0x7fbd08106af0 2024-09-19T06:10:19.119 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.367+0000 7fbd0ccb6640 1 -- 172.21.15.53:0/3187933874 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fbd04030620 con 0x7fbd08106af0 2024-09-19T06:10:19.119 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.368+0000 7fbd0ff1c640 1 -- 172.21.15.53:0/3187933874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd08106af0 msgr2=0x7fbd08106ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:19.119 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.368+0000 7fbd0ff1c640 1 --2- 172.21.15.53:0/3187933874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd08106af0 0x7fbd08106ef0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbd04009820 tx=0x7fbd0402a270 comp rx=0 tx=0).stop 2024-09-19T06:10:19.119 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.368+0000 7fbd0ff1c640 1 -- 172.21.15.53:0/3187933874 shutdown_connections 2024-09-19T06:10:19.119 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.368+0000 7fbd0ff1c640 1 --2- 172.21.15.53:0/3187933874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd08106af0 0x7fbd08106ef0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.119 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.368+0000 7fbd0ff1c640 1 -- 172.21.15.53:0/3187933874 >> 172.21.15.53:0/3187933874 conn(0x7fbd081023c0 msgr2=0x7fbd081047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:19.119 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.368+0000 7fbd0ff1c640 1 -- 172.21.15.53:0/3187933874 shutdown_connections 2024-09-19T06:10:19.119 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.368+0000 7fbd0ff1c640 1 -- 172.21.15.53:0/3187933874 wait complete. 2024-09-19T06:10:19.119 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.369+0000 7fbd0ff1c640 1 Processor -- start 2024-09-19T06:10:19.119 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.369+0000 7fbd0ff1c640 1 -- start start 2024-09-19T06:10:19.119 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.369+0000 7fbd0ff1c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd08106af0 0x7fbd0819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:19.120 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.369+0000 7fbd0ff1c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0810af60 con 0x7fbd08106af0 2024-09-19T06:10:19.120 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.369+0000 7fbd0dcb8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd08106af0 0x7fbd0819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:19.120 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.369+0000 7fbd0dcb8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd08106af0 0x7fbd0819aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47920/0 (socket says 172.21.15.53:47920) 2024-09-19T06:10:19.120 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.369+0000 7fbd0dcb8640 1 -- 172.21.15.53:0/3053197654 learned_addr learned my addr 172.21.15.53:0/3053197654 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:19.120 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.369+0000 7fbd0dcb8640 1 -- 172.21.15.53:0/3053197654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd0819af50 con 0x7fbd08106af0 2024-09-19T06:10:19.120 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.370+0000 7fbd0dcb8640 1 --2- 172.21.15.53:0/3053197654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd08106af0 0x7fbd0819aa10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbd04006f10 tx=0x7fbd04032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:19.120 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.370+0000 7fbcfeffd640 1 -- 172.21.15.53:0/3053197654 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fbd04032a70 con 0x7fbd08106af0 2024-09-19T06:10:19.120 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.370+0000 7fbcfeffd640 1 -- 172.21.15.53:0/3053197654 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd04032c10 con 0x7fbd08106af0 2024-09-19T06:10:19.120 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.370+0000 7fbd0ff1c640 1 -- 172.21.15.53:0/3053197654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd0819b1e0 con 0x7fbd08106af0 2024-09-19T06:10:19.120 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.370+0000 7fbcfeffd640 1 -- 172.21.15.53:0/3053197654 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fbd0403d620 con 0x7fbd08106af0 2024-09-19T06:10:19.120 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.370+0000 7fbd0ff1c640 1 -- 172.21.15.53:0/3053197654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd0819ded0 con 0x7fbd08106af0 2024-09-19T06:10:19.120 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.371+0000 7fbcfeffd640 1 -- 172.21.15.53:0/3053197654 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) ==== 46476+0+0 (secure 0 0 0) 0x7fbd0403d8d0 con 0x7fbd08106af0 2024-09-19T06:10:19.120 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.371+0000 7fbcfeffd640 1 --2- 172.21.15.53:0/3053197654 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fbcf4039230 0x7fbcf403b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:19.120 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.371+0000 7fbcfeffd640 1 -- 172.21.15.53:0/3053197654 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbd0406bd60 con 0x7fbd08106af0 2024-09-19T06:10:19.121 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.371+0000 7fbd0ff1c640 1 -- 172.21.15.53:0/3053197654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd08106ef0 con 0x7fbd08106af0 2024-09-19T06:10:19.121 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.371+0000 7fbd0d4b7640 1 --2- 172.21.15.53:0/3053197654 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fbcf4039230 0x7fbcf403b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:19.121 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.372+0000 7fbd0d4b7640 1 --2- 172.21.15.53:0/3053197654 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fbcf4039230 0x7fbcf403b6f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80098c0 tx=0x7fbcf8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:19.121 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.376+0000 7fbcfeffd640 1 -- 172.21.15.53:0/3053197654 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbd0406bac0 con 0x7fbd08106af0 2024-09-19T06:10:19.121 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.567+0000 7fbd0ff1c640 1 -- 172.21.15.53:0/3053197654 --> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) -- 0x7fbd081035a0 con 0x7fbcf4039230 2024-09-19T06:10:19.121 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.571+0000 7fbcfeffd640 1 -- 172.21.15.53:0/3053197654 <== mgr.14118 v2:172.21.15.53:6800/681468213 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fbd081035a0 con 0x7fbcf4039230 2024-09-19T06:10:19.121 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.573+0000 7fbd0ff1c640 1 -- 172.21.15.53:0/3053197654 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fbcf4039230 msgr2=0x7fbcf403b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:19.121 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.573+0000 7fbd0ff1c640 1 --2- 172.21.15.53:0/3053197654 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fbcf4039230 0x7fbcf403b6f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80098c0 tx=0x7fbcf8006db0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.121 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.573+0000 7fbd0ff1c640 1 -- 172.21.15.53:0/3053197654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd08106af0 msgr2=0x7fbd0819aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:19.121 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.573+0000 7fbd0ff1c640 1 --2- 172.21.15.53:0/3053197654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd08106af0 0x7fbd0819aa10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbd04006f10 tx=0x7fbd04032670 comp rx=0 tx=0).stop 2024-09-19T06:10:19.121 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.573+0000 7fbd0ff1c640 1 -- 172.21.15.53:0/3053197654 shutdown_connections 2024-09-19T06:10:19.121 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.573+0000 7fbd0ff1c640 1 --2- 172.21.15.53:0/3053197654 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fbcf4039230 0x7fbcf403b6f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.121 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.573+0000 7fbd0ff1c640 1 --2- 172.21.15.53:0/3053197654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd08106af0 0x7fbd0819aa10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.122 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.573+0000 7fbd0ff1c640 1 -- 172.21.15.53:0/3053197654 >> 172.21.15.53:0/3053197654 conn(0x7fbd081023c0 msgr2=0x7fbd08102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:19.122 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.573+0000 7fbd0ff1c640 1 -- 172.21.15.53:0/3053197654 shutdown_connections 2024-09-19T06:10:19.122 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:18.573+0000 7fbd0ff1c640 1 -- 172.21.15.53:0/3053197654 wait complete. 2024-09-19T06:10:19.122 INFO:teuthology.orchestra.run.smithi053.stdout:Deploying node-exporter service with default placement... 2024-09-19T06:10:19.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:19 smithi053 ceph-mon[27968]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:10:19.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:19 smithi053 ceph-mon[27968]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:10:19.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:19 smithi053 ceph-mon[27968]: Saving service grafana spec with placement count:1 2024-09-19T06:10:19.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:19 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:20.340 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-09-19T06:10:20.340 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.521+0000 7fbb6c70e640 1 Processor -- start 2024-09-19T06:10:20.340 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.521+0000 7fbb6c70e640 1 -- start start 2024-09-19T06:10:20.340 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.521+0000 7fbb6c70e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb64106af0 0x7fbb64106ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:20.341 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.521+0000 7fbb6c70e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbb64107430 con 0x7fbb64106af0 2024-09-19T06:10:20.341 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.521+0000 7fbb6a4aa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb64106af0 0x7fbb64106ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:20.341 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.521+0000 7fbb6a4aa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb64106af0 0x7fbb64106ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47926/0 (socket says 172.21.15.53:47926) 2024-09-19T06:10:20.341 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.521+0000 7fbb6a4aa640 1 -- 172.21.15.53:0/2186123844 learned_addr learned my addr 172.21.15.53:0/2186123844 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:20.341 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.522+0000 7fbb6a4aa640 1 -- 172.21.15.53:0/2186123844 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb641075b0 con 0x7fbb64106af0 2024-09-19T06:10:20.341 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.522+0000 7fbb6a4aa640 1 --2- 172.21.15.53:0/2186123844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb64106af0 0x7fbb64106ef0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbb60009820 tx=0x7fbb6002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e86f1eba0f1144 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:20.341 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.523+0000 7fbb694a8640 1 -- 172.21.15.53:0/2186123844 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fbb60037070 con 0x7fbb64106af0 2024-09-19T06:10:20.341 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.523+0000 7fbb694a8640 1 -- 172.21.15.53:0/2186123844 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb6002ad40 con 0x7fbb64106af0 2024-09-19T06:10:20.341 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.523+0000 7fbb694a8640 1 -- 172.21.15.53:0/2186123844 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fbb60030620 con 0x7fbb64106af0 2024-09-19T06:10:20.341 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.523+0000 7fbb6c70e640 1 -- 172.21.15.53:0/2186123844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb64106af0 msgr2=0x7fbb64106ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:20.341 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.523+0000 7fbb6c70e640 1 --2- 172.21.15.53:0/2186123844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb64106af0 0x7fbb64106ef0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbb60009820 tx=0x7fbb6002a270 comp rx=0 tx=0).stop 2024-09-19T06:10:20.341 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.523+0000 7fbb6c70e640 1 -- 172.21.15.53:0/2186123844 shutdown_connections 2024-09-19T06:10:20.341 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.523+0000 7fbb6c70e640 1 --2- 172.21.15.53:0/2186123844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb64106af0 0x7fbb64106ef0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:20.342 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.523+0000 7fbb6c70e640 1 -- 172.21.15.53:0/2186123844 >> 172.21.15.53:0/2186123844 conn(0x7fbb641023c0 msgr2=0x7fbb641047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:20.342 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.523+0000 7fbb6c70e640 1 -- 172.21.15.53:0/2186123844 shutdown_connections 2024-09-19T06:10:20.342 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.523+0000 7fbb6c70e640 1 -- 172.21.15.53:0/2186123844 wait complete. 2024-09-19T06:10:20.342 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.524+0000 7fbb6c70e640 1 Processor -- start 2024-09-19T06:10:20.342 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.524+0000 7fbb6c70e640 1 -- start start 2024-09-19T06:10:20.342 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.524+0000 7fbb6c70e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb64106af0 0x7fbb6419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:20.342 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.524+0000 7fbb6c70e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6410af60 con 0x7fbb64106af0 2024-09-19T06:10:20.342 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.524+0000 7fbb6a4aa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb64106af0 0x7fbb6419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:20.342 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.524+0000 7fbb6a4aa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb64106af0 0x7fbb6419aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47932/0 (socket says 172.21.15.53:47932) 2024-09-19T06:10:20.342 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.524+0000 7fbb6a4aa640 1 -- 172.21.15.53:0/2131708057 learned_addr learned my addr 172.21.15.53:0/2131708057 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:20.342 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.525+0000 7fbb6a4aa640 1 -- 172.21.15.53:0/2131708057 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb6419af50 con 0x7fbb64106af0 2024-09-19T06:10:20.342 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.525+0000 7fbb6a4aa640 1 --2- 172.21.15.53:0/2131708057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb64106af0 0x7fbb6419aa10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbb60006f10 tx=0x7fbb60032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:20.342 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.525+0000 7fbb5b7fe640 1 -- 172.21.15.53:0/2131708057 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fbb60032a70 con 0x7fbb64106af0 2024-09-19T06:10:20.342 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.525+0000 7fbb5b7fe640 1 -- 172.21.15.53:0/2131708057 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb60032c10 con 0x7fbb64106af0 2024-09-19T06:10:20.343 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.525+0000 7fbb6c70e640 1 -- 172.21.15.53:0/2131708057 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb6419b1e0 con 0x7fbb64106af0 2024-09-19T06:10:20.343 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.525+0000 7fbb5b7fe640 1 -- 172.21.15.53:0/2131708057 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fbb6003d620 con 0x7fbb64106af0 2024-09-19T06:10:20.343 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.525+0000 7fbb6c70e640 1 -- 172.21.15.53:0/2131708057 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb6419ded0 con 0x7fbb64106af0 2024-09-19T06:10:20.343 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.526+0000 7fbb5b7fe640 1 -- 172.21.15.53:0/2131708057 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) ==== 46476+0+0 (secure 0 0 0) 0x7fbb6003d8d0 con 0x7fbb64106af0 2024-09-19T06:10:20.343 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.526+0000 7fbb6c70e640 1 -- 172.21.15.53:0/2131708057 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb64106ef0 con 0x7fbb64106af0 2024-09-19T06:10:20.343 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.526+0000 7fbb5b7fe640 1 --2- 172.21.15.53:0/2131708057 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fbb50039230 0x7fbb5003b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:20.343 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.526+0000 7fbb5b7fe640 1 -- 172.21.15.53:0/2131708057 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbb6006bd10 con 0x7fbb64106af0 2024-09-19T06:10:20.343 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.526+0000 7fbb69ca9640 1 --2- 172.21.15.53:0/2131708057 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fbb50039230 0x7fbb5003b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:20.343 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.527+0000 7fbb69ca9640 1 --2- 172.21.15.53:0/2131708057 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fbb50039230 0x7fbb5003b6f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbb540098c0 tx=0x7fbb54006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:20.343 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.531+0000 7fbb5b7fe640 1 -- 172.21.15.53:0/2131708057 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbb6002e200 con 0x7fbb64106af0 2024-09-19T06:10:20.343 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.725+0000 7fbb6c70e640 1 -- 172.21.15.53:0/2131708057 --> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) -- 0x7fbb641035a0 con 0x7fbb50039230 2024-09-19T06:10:20.343 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.729+0000 7fbb5b7fe640 1 -- 172.21.15.53:0/2131708057 <== mgr.14118 v2:172.21.15.53:6800/681468213 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+34 (secure 0 0 0) 0x7fbb641035a0 con 0x7fbb50039230 2024-09-19T06:10:20.343 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.732+0000 7fbb6c70e640 1 -- 172.21.15.53:0/2131708057 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fbb50039230 msgr2=0x7fbb5003b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:20.343 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.732+0000 7fbb6c70e640 1 --2- 172.21.15.53:0/2131708057 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fbb50039230 0x7fbb5003b6f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbb540098c0 tx=0x7fbb54006db0 comp rx=0 tx=0).stop 2024-09-19T06:10:20.344 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.732+0000 7fbb6c70e640 1 -- 172.21.15.53:0/2131708057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb64106af0 msgr2=0x7fbb6419aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:20.344 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.732+0000 7fbb6c70e640 1 --2- 172.21.15.53:0/2131708057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb64106af0 0x7fbb6419aa10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbb60006f10 tx=0x7fbb60032670 comp rx=0 tx=0).stop 2024-09-19T06:10:20.344 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.732+0000 7fbb6c70e640 1 -- 172.21.15.53:0/2131708057 shutdown_connections 2024-09-19T06:10:20.344 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.732+0000 7fbb6c70e640 1 --2- 172.21.15.53:0/2131708057 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fbb50039230 0x7fbb5003b6f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:20.344 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.732+0000 7fbb6c70e640 1 --2- 172.21.15.53:0/2131708057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb64106af0 0x7fbb6419aa10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:20.344 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.732+0000 7fbb6c70e640 1 -- 172.21.15.53:0/2131708057 >> 172.21.15.53:0/2131708057 conn(0x7fbb641023c0 msgr2=0x7fbb64102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:20.344 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.732+0000 7fbb6c70e640 1 -- 172.21.15.53:0/2131708057 shutdown_connections 2024-09-19T06:10:20.344 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:19.732+0000 7fbb6c70e640 1 -- 172.21.15.53:0/2131708057 wait complete. 2024-09-19T06:10:20.344 INFO:teuthology.orchestra.run.smithi053.stdout:Deploying alertmanager service with default placement... 2024-09-19T06:10:20.939 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:20 smithi053 ceph-mon[27968]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:10:20.939 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:20 smithi053 ceph-mon[27968]: Saving service node-exporter spec with placement * 2024-09-19T06:10:20.939 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:20 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:21.496 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-09-19T06:10:21.496 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.737+0000 7fcae666d640 1 Processor -- start 2024-09-19T06:10:21.496 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.737+0000 7fcae666d640 1 -- start start 2024-09-19T06:10:21.496 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.737+0000 7fcae666d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcae0107550 0x7fcae0109940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:21.497 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.737+0000 7fcae666d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcae007a730 con 0x7fcae0107550 2024-09-19T06:10:21.497 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.737+0000 7fcadffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcae0107550 0x7fcae0109940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:21.497 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.737+0000 7fcadffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcae0107550 0x7fcae0109940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47938/0 (socket says 172.21.15.53:47938) 2024-09-19T06:10:21.497 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.737+0000 7fcadffff640 1 -- 172.21.15.53:0/2798655323 learned_addr learned my addr 172.21.15.53:0/2798655323 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:21.497 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.737+0000 7fcadffff640 1 -- 172.21.15.53:0/2798655323 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae0109e80 con 0x7fcae0107550 2024-09-19T06:10:21.497 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.738+0000 7fcadffff640 1 --2- 172.21.15.53:0/2798655323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcae0107550 0x7fcae0109940 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcad0009820 tx=0x7fcad002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3481bfeb92b7b5bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:21.497 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.738+0000 7fcadeffd640 1 -- 172.21.15.53:0/2798655323 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fcad0037070 con 0x7fcae0107550 2024-09-19T06:10:21.497 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.738+0000 7fcadeffd640 1 -- 172.21.15.53:0/2798655323 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcad002ad40 con 0x7fcae0107550 2024-09-19T06:10:21.497 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.738+0000 7fcadeffd640 1 -- 172.21.15.53:0/2798655323 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fcad0030620 con 0x7fcae0107550 2024-09-19T06:10:21.497 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.739+0000 7fcae666d640 1 -- 172.21.15.53:0/2798655323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcae0107550 msgr2=0x7fcae0109940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:21.497 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.739+0000 7fcae666d640 1 --2- 172.21.15.53:0/2798655323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcae0107550 0x7fcae0109940 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcad0009820 tx=0x7fcad002a270 comp rx=0 tx=0).stop 2024-09-19T06:10:21.497 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.739+0000 7fcae666d640 1 -- 172.21.15.53:0/2798655323 shutdown_connections 2024-09-19T06:10:21.497 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.739+0000 7fcae666d640 1 --2- 172.21.15.53:0/2798655323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcae0107550 0x7fcae0109940 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:21.498 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.739+0000 7fcae666d640 1 -- 172.21.15.53:0/2798655323 >> 172.21.15.53:0/2798655323 conn(0x7fcae0101150 msgr2=0x7fcae01035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:21.498 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.739+0000 7fcae666d640 1 -- 172.21.15.53:0/2798655323 shutdown_connections 2024-09-19T06:10:21.498 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.739+0000 7fcae666d640 1 -- 172.21.15.53:0/2798655323 wait complete. 2024-09-19T06:10:21.498 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.740+0000 7fcae666d640 1 Processor -- start 2024-09-19T06:10:21.498 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.740+0000 7fcae666d640 1 -- start start 2024-09-19T06:10:21.498 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.740+0000 7fcae666d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcae0107550 0x7fcae019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:21.498 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.740+0000 7fcae666d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcae010af00 con 0x7fcae0107550 2024-09-19T06:10:21.498 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.740+0000 7fcadffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcae0107550 0x7fcae019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:21.498 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.740+0000 7fcadffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcae0107550 0x7fcae019aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47948/0 (socket says 172.21.15.53:47948) 2024-09-19T06:10:21.498 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.740+0000 7fcadffff640 1 -- 172.21.15.53:0/3825621476 learned_addr learned my addr 172.21.15.53:0/3825621476 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:21.498 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.740+0000 7fcadffff640 1 -- 172.21.15.53:0/3825621476 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae019afa0 con 0x7fcae0107550 2024-09-19T06:10:21.498 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.741+0000 7fcadffff640 1 --2- 172.21.15.53:0/3825621476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcae0107550 0x7fcae019aa60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcad0000c00 tx=0x7fcad0030fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:21.498 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.741+0000 7fcadd7fa640 1 -- 172.21.15.53:0/3825621476 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fcad0032a00 con 0x7fcae0107550 2024-09-19T06:10:21.499 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.741+0000 7fcadd7fa640 1 -- 172.21.15.53:0/3825621476 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcad0032ba0 con 0x7fcae0107550 2024-09-19T06:10:21.499 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.741+0000 7fcae666d640 1 -- 172.21.15.53:0/3825621476 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcae019b230 con 0x7fcae0107550 2024-09-19T06:10:21.499 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.741+0000 7fcadd7fa640 1 -- 172.21.15.53:0/3825621476 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fcad003d5d0 con 0x7fcae0107550 2024-09-19T06:10:21.499 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.741+0000 7fcae666d640 1 -- 172.21.15.53:0/3825621476 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcae019df20 con 0x7fcae0107550 2024-09-19T06:10:21.499 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.742+0000 7fcadd7fa640 1 -- 172.21.15.53:0/3825621476 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) ==== 46476+0+0 (secure 0 0 0) 0x7fcad003e430 con 0x7fcae0107550 2024-09-19T06:10:21.499 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.742+0000 7fcae666d640 1 -- 172.21.15.53:0/3825621476 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcae01079b0 con 0x7fcae0107550 2024-09-19T06:10:21.499 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.742+0000 7fcadd7fa640 1 --2- 172.21.15.53:0/3825621476 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fcac8039230 0x7fcac803b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:21.499 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.742+0000 7fcadd7fa640 1 -- 172.21.15.53:0/3825621476 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcad006be00 con 0x7fcae0107550 2024-09-19T06:10:21.499 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.742+0000 7fcadf7fe640 1 --2- 172.21.15.53:0/3825621476 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fcac8039230 0x7fcac803b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:21.499 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.743+0000 7fcadf7fe640 1 --2- 172.21.15.53:0/3825621476 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fcac8039230 0x7fcac803b6f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcad40098c0 tx=0x7fcad4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:21.499 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.748+0000 7fcadd7fa640 1 -- 172.21.15.53:0/3825621476 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcad003e720 con 0x7fcae0107550 2024-09-19T06:10:21.499 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.936+0000 7fcae666d640 1 -- 172.21.15.53:0/3825621476 --> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) -- 0x7fcae0102430 con 0x7fcac8039230 2024-09-19T06:10:21.499 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.941+0000 7fcadd7fa640 1 -- 172.21.15.53:0/3825621476 <== mgr.14118 v2:172.21.15.53:6800/681468213 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+33 (secure 0 0 0) 0x7fcae0102430 con 0x7fcac8039230 2024-09-19T06:10:21.500 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.943+0000 7fcae666d640 1 -- 172.21.15.53:0/3825621476 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fcac8039230 msgr2=0x7fcac803b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:21.500 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.943+0000 7fcae666d640 1 --2- 172.21.15.53:0/3825621476 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fcac8039230 0x7fcac803b6f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcad40098c0 tx=0x7fcad4006db0 comp rx=0 tx=0).stop 2024-09-19T06:10:21.500 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.943+0000 7fcae666d640 1 -- 172.21.15.53:0/3825621476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcae0107550 msgr2=0x7fcae019aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:21.500 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.943+0000 7fcae666d640 1 --2- 172.21.15.53:0/3825621476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcae0107550 0x7fcae019aa60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcad0000c00 tx=0x7fcad0030fc0 comp rx=0 tx=0).stop 2024-09-19T06:10:21.500 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.943+0000 7fcae666d640 1 -- 172.21.15.53:0/3825621476 shutdown_connections 2024-09-19T06:10:21.500 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.943+0000 7fcae666d640 1 --2- 172.21.15.53:0/3825621476 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fcac8039230 0x7fcac803b6f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:21.500 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.943+0000 7fcae666d640 1 --2- 172.21.15.53:0/3825621476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcae0107550 0x7fcae019aa60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:21.500 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.943+0000 7fcae666d640 1 -- 172.21.15.53:0/3825621476 >> 172.21.15.53:0/3825621476 conn(0x7fcae0101150 msgr2=0x7fcae0101d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:21.500 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.943+0000 7fcae666d640 1 -- 172.21.15.53:0/3825621476 shutdown_connections 2024-09-19T06:10:21.500 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:20.943+0000 7fcae666d640 1 -- 172.21.15.53:0/3825621476 wait complete. 2024-09-19T06:10:22.098 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:21 smithi053 ceph-mon[27968]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:10:22.098 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:21 smithi053 ceph-mon[27968]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:10:22.099 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:21 smithi053 ceph-mon[27968]: Saving service alertmanager spec with placement count:1 2024-09-19T06:10:22.099 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:21 smithi053 ceph-mon[27968]: from='mgr.14118 172.21.15.53:0/2304583254' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:22.719 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.910+0000 7f295d5d4640 1 Processor -- start 2024-09-19T06:10:22.719 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.910+0000 7f295d5d4640 1 -- start start 2024-09-19T06:10:22.719 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.910+0000 7f295d5d4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2958105390 0x7f2958105790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:22.719 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.910+0000 7f295d5d4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f295807a6d0 con 0x7f2958105390 2024-09-19T06:10:22.719 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.911+0000 7f2956ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2958105390 0x7f2958105790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:22.719 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.911+0000 7f2956ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2958105390 0x7f2958105790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44444/0 (socket says 172.21.15.53:44444) 2024-09-19T06:10:22.719 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.911+0000 7f2956ffd640 1 -- 172.21.15.53:0/722622566 learned_addr learned my addr 172.21.15.53:0/722622566 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:22.719 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.911+0000 7f2956ffd640 1 -- 172.21.15.53:0/722622566 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2958105cd0 con 0x7f2958105390 2024-09-19T06:10:22.720 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.912+0000 7f2956ffd640 1 --2- 172.21.15.53:0/722622566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2958105390 0x7f2958105790 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2948009820 tx=0x7f294802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc222bde5e5657fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:22.720 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.912+0000 7f29567fc640 1 -- 172.21.15.53:0/722622566 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f2948037070 con 0x7f2958105390 2024-09-19T06:10:22.720 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.912+0000 7f29567fc640 1 -- 172.21.15.53:0/722622566 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f294802ad40 con 0x7f2958105390 2024-09-19T06:10:22.720 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.912+0000 7f29567fc640 1 -- 172.21.15.53:0/722622566 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f2948030620 con 0x7f2958105390 2024-09-19T06:10:22.720 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.913+0000 7f295d5d4640 1 -- 172.21.15.53:0/722622566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2958105390 msgr2=0x7f2958105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:22.720 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.913+0000 7f295d5d4640 1 --2- 172.21.15.53:0/722622566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2958105390 0x7f2958105790 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2948009820 tx=0x7f294802a270 comp rx=0 tx=0).stop 2024-09-19T06:10:22.720 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.913+0000 7f295d5d4640 1 -- 172.21.15.53:0/722622566 shutdown_connections 2024-09-19T06:10:22.720 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.913+0000 7f295d5d4640 1 --2- 172.21.15.53:0/722622566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2958105390 0x7f2958105790 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:22.720 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.913+0000 7f295d5d4640 1 -- 172.21.15.53:0/722622566 >> 172.21.15.53:0/722622566 conn(0x7f29581011a0 msgr2=0x7f29581035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:22.720 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.913+0000 7f295d5d4640 1 -- 172.21.15.53:0/722622566 shutdown_connections 2024-09-19T06:10:22.720 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.913+0000 7f295d5d4640 1 -- 172.21.15.53:0/722622566 wait complete. 2024-09-19T06:10:22.720 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.913+0000 7f295d5d4640 1 Processor -- start 2024-09-19T06:10:22.721 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.913+0000 7f295d5d4640 1 -- start start 2024-09-19T06:10:22.721 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.914+0000 7f295d5d4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2958105390 0x7f295819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:22.721 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.914+0000 7f295d5d4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2958106db0 con 0x7f2958105390 2024-09-19T06:10:22.721 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.914+0000 7f2956ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2958105390 0x7f295819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:22.721 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.914+0000 7f2956ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2958105390 0x7f295819cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44458/0 (socket says 172.21.15.53:44458) 2024-09-19T06:10:22.721 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.914+0000 7f2956ffd640 1 -- 172.21.15.53:0/3164857965 learned_addr learned my addr 172.21.15.53:0/3164857965 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:22.721 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.914+0000 7f2956ffd640 1 -- 172.21.15.53:0/3164857965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f295819d1a0 con 0x7f2958105390 2024-09-19T06:10:22.721 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.914+0000 7f2956ffd640 1 --2- 172.21.15.53:0/3164857965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2958105390 0x7f295819cc60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2948000c00 tx=0x7f2948030fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:22.722 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.914+0000 7f2954ff9640 1 -- 172.21.15.53:0/3164857965 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f2948032a00 con 0x7f2958105390 2024-09-19T06:10:22.722 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.914+0000 7f2954ff9640 1 -- 172.21.15.53:0/3164857965 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2948032ba0 con 0x7f2958105390 2024-09-19T06:10:22.722 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.914+0000 7f295d5d4640 1 -- 172.21.15.53:0/3164857965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f295819d430 con 0x7f2958105390 2024-09-19T06:10:22.722 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.914+0000 7f2954ff9640 1 -- 172.21.15.53:0/3164857965 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f294803d5d0 con 0x7f2958105390 2024-09-19T06:10:22.722 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.914+0000 7f295d5d4640 1 -- 172.21.15.53:0/3164857965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29581a0120 con 0x7f2958105390 2024-09-19T06:10:22.722 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.915+0000 7f2954ff9640 1 -- 172.21.15.53:0/3164857965 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) ==== 46476+0+0 (secure 0 0 0) 0x7f294803e430 con 0x7f2958105390 2024-09-19T06:10:22.722 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.915+0000 7f295d5d4640 1 -- 172.21.15.53:0/3164857965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2958105790 con 0x7f2958105390 2024-09-19T06:10:22.722 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.916+0000 7f2954ff9640 1 --2- 172.21.15.53:0/3164857965 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f2944039230 0x7f294403b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:22.722 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.916+0000 7f2954ff9640 1 -- 172.21.15.53:0/3164857965 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f294806be00 con 0x7f2958105390 2024-09-19T06:10:22.722 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.916+0000 7f294ffff640 1 --2- 172.21.15.53:0/3164857965 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f2944039230 0x7f294403b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:22.722 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.916+0000 7f294ffff640 1 --2- 172.21.15.53:0/3164857965 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f2944039230 0x7f294403b6f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f29400098c0 tx=0x7f2940006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:22.722 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:21.921+0000 7f2954ff9640 1 -- 172.21.15.53:0/3164857965 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f294803e720 con 0x7f2958105390 2024-09-19T06:10:22.722 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:22.096+0000 7f295d5d4640 1 -- 172.21.15.53:0/3164857965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f29581960d0 con 0x7f2958105390 2024-09-19T06:10:22.723 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:22.099+0000 7f2954ff9640 1 -- 172.21.15.53:0/3164857965 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f2948031e80 con 0x7f2958105390 2024-09-19T06:10:22.723 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:22.101+0000 7f295d5d4640 1 -- 172.21.15.53:0/3164857965 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f2944039230 msgr2=0x7f294403b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:22.723 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:22.101+0000 7f295d5d4640 1 --2- 172.21.15.53:0/3164857965 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f2944039230 0x7f294403b6f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f29400098c0 tx=0x7f2940006db0 comp rx=0 tx=0).stop 2024-09-19T06:10:22.723 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:22.101+0000 7f295d5d4640 1 -- 172.21.15.53:0/3164857965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2958105390 msgr2=0x7f295819cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:22.723 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:22.101+0000 7f295d5d4640 1 --2- 172.21.15.53:0/3164857965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2958105390 0x7f295819cc60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2948000c00 tx=0x7f2948030fc0 comp rx=0 tx=0).stop 2024-09-19T06:10:22.723 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:22.101+0000 7f295d5d4640 1 -- 172.21.15.53:0/3164857965 shutdown_connections 2024-09-19T06:10:22.723 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:22.101+0000 7f295d5d4640 1 --2- 172.21.15.53:0/3164857965 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f2944039230 0x7f294403b6f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:22.723 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:22.101+0000 7f295d5d4640 1 --2- 172.21.15.53:0/3164857965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2958105390 0x7f295819cc60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:22.724 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:22.101+0000 7f295d5d4640 1 -- 172.21.15.53:0/3164857965 >> 172.21.15.53:0/3164857965 conn(0x7f29581011a0 msgr2=0x7f2958190f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:22.724 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:22.101+0000 7f295d5d4640 1 -- 172.21.15.53:0/3164857965 shutdown_connections 2024-09-19T06:10:22.724 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:22.101+0000 7f295d5d4640 1 -- 172.21.15.53:0/3164857965 wait complete. 2024-09-19T06:10:23.250 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:23 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/3164857965' entity='client.admin' 2024-09-19T06:10:23.808 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.062+0000 7fed0c83f640 1 Processor -- start 2024-09-19T06:10:23.808 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.062+0000 7fed0c83f640 1 -- start start 2024-09-19T06:10:23.809 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.063+0000 7fed0c83f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed08106ac0 0x7fed08106ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:23.809 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.063+0000 7fed0c83f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fed08107400 con 0x7fed08106ac0 2024-09-19T06:10:23.809 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.063+0000 7fed0659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed08106ac0 0x7fed08106ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:23.809 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.063+0000 7fed0659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed08106ac0 0x7fed08106ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44468/0 (socket says 172.21.15.53:44468) 2024-09-19T06:10:23.809 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.063+0000 7fed0659c640 1 -- 172.21.15.53:0/1490408906 learned_addr learned my addr 172.21.15.53:0/1490408906 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:23.809 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.063+0000 7fed0659c640 1 -- 172.21.15.53:0/1490408906 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed08107580 con 0x7fed08106ac0 2024-09-19T06:10:23.809 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.064+0000 7fed0659c640 1 --2- 172.21.15.53:0/1490408906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed08106ac0 0x7fed08106ec0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009820 tx=0x7fecf802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84f05d9b6da0fdb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:23.809 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.064+0000 7fed0559a640 1 -- 172.21.15.53:0/1490408906 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fecf8037070 con 0x7fed08106ac0 2024-09-19T06:10:23.809 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.064+0000 7fed0559a640 1 -- 172.21.15.53:0/1490408906 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecf802ad40 con 0x7fed08106ac0 2024-09-19T06:10:23.809 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.064+0000 7fed0559a640 1 -- 172.21.15.53:0/1490408906 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fecf8030620 con 0x7fed08106ac0 2024-09-19T06:10:23.809 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.065+0000 7fed0c83f640 1 -- 172.21.15.53:0/1490408906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed08106ac0 msgr2=0x7fed08106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:23.809 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.065+0000 7fed0c83f640 1 --2- 172.21.15.53:0/1490408906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed08106ac0 0x7fed08106ec0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009820 tx=0x7fecf802a270 comp rx=0 tx=0).stop 2024-09-19T06:10:23.809 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.065+0000 7fed0c83f640 1 -- 172.21.15.53:0/1490408906 shutdown_connections 2024-09-19T06:10:23.810 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.065+0000 7fed0c83f640 1 --2- 172.21.15.53:0/1490408906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed08106ac0 0x7fed08106ec0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.810 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.065+0000 7fed0c83f640 1 -- 172.21.15.53:0/1490408906 >> 172.21.15.53:0/1490408906 conn(0x7fed08102330 msgr2=0x7fed08104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:23.810 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.065+0000 7fed0c83f640 1 -- 172.21.15.53:0/1490408906 shutdown_connections 2024-09-19T06:10:23.810 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.065+0000 7fed0c83f640 1 -- 172.21.15.53:0/1490408906 wait complete. 2024-09-19T06:10:23.810 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.066+0000 7fed0c83f640 1 Processor -- start 2024-09-19T06:10:23.810 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.066+0000 7fed0c83f640 1 -- start start 2024-09-19T06:10:23.810 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.066+0000 7fed0c83f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed08106ac0 0x7fed0819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:23.810 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.066+0000 7fed0c83f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fed0810af30 con 0x7fed08106ac0 2024-09-19T06:10:23.810 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.066+0000 7fed0659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed08106ac0 0x7fed0819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:23.810 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.066+0000 7fed0659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed08106ac0 0x7fed0819aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44484/0 (socket says 172.21.15.53:44484) 2024-09-19T06:10:23.811 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.066+0000 7fed0659c640 1 -- 172.21.15.53:0/2858815569 learned_addr learned my addr 172.21.15.53:0/2858815569 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:23.811 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.066+0000 7fed0659c640 1 -- 172.21.15.53:0/2858815569 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0819af40 con 0x7fed08106ac0 2024-09-19T06:10:23.811 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.067+0000 7fed0659c640 1 --2- 172.21.15.53:0/2858815569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed08106ac0 0x7fed0819aa00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fecf8006f10 tx=0x7fecf8032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:23.811 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.067+0000 7fecf77fe640 1 -- 172.21.15.53:0/2858815569 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fecf8032a70 con 0x7fed08106ac0 2024-09-19T06:10:23.811 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.067+0000 7fed0c83f640 1 -- 172.21.15.53:0/2858815569 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed0819b1d0 con 0x7fed08106ac0 2024-09-19T06:10:23.811 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.067+0000 7fecf77fe640 1 -- 172.21.15.53:0/2858815569 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecf8032c10 con 0x7fed08106ac0 2024-09-19T06:10:23.811 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.067+0000 7fecf77fe640 1 -- 172.21.15.53:0/2858815569 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fecf803d620 con 0x7fed08106ac0 2024-09-19T06:10:23.811 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.067+0000 7fed0c83f640 1 -- 172.21.15.53:0/2858815569 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed0819dec0 con 0x7fed08106ac0 2024-09-19T06:10:23.811 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.068+0000 7fecf77fe640 1 -- 172.21.15.53:0/2858815569 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) ==== 46476+0+0 (secure 0 0 0) 0x7fecf803d920 con 0x7fed08106ac0 2024-09-19T06:10:23.812 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.068+0000 7fecf77fe640 1 --2- 172.21.15.53:0/2858815569 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fecf00391e0 0x7fecf003b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:23.812 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.068+0000 7fecf77fe640 1 -- 172.21.15.53:0/2858815569 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fecf806bdd0 con 0x7fed08106ac0 2024-09-19T06:10:23.812 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.068+0000 7fed05d9b640 1 --2- 172.21.15.53:0/2858815569 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fecf00391e0 0x7fecf003b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:23.812 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.068+0000 7fed0c83f640 1 -- 172.21.15.53:0/2858815569 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed08106ec0 con 0x7fed08106ac0 2024-09-19T06:10:23.812 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.068+0000 7fed05d9b640 1 --2- 172.21.15.53:0/2858815569 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fecf00391e0 0x7fecf003b6a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fecfc0098c0 tx=0x7fecfc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:23.812 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.073+0000 7fecf77fe640 1 -- 172.21.15.53:0/2858815569 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fecf80454d0 con 0x7fed08106ac0 2024-09-19T06:10:23.812 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.248+0000 7fed0c83f640 1 -- 172.21.15.53:0/2858815569 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fed0819e090 con 0x7fed08106ac0 2024-09-19T06:10:23.812 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.250+0000 7fecf77fe640 1 -- 172.21.15.53:0/2858815569 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fecf8045730 con 0x7fed08106ac0 2024-09-19T06:10:23.812 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.253+0000 7fed0c83f640 1 -- 172.21.15.53:0/2858815569 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fecf00391e0 msgr2=0x7fecf003b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:23.812 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.253+0000 7fed0c83f640 1 --2- 172.21.15.53:0/2858815569 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fecf00391e0 0x7fecf003b6a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fecfc0098c0 tx=0x7fecfc006db0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.812 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.253+0000 7fed0c83f640 1 -- 172.21.15.53:0/2858815569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed08106ac0 msgr2=0x7fed0819aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:23.812 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.253+0000 7fed0c83f640 1 --2- 172.21.15.53:0/2858815569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed08106ac0 0x7fed0819aa00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fecf8006f10 tx=0x7fecf8032670 comp rx=0 tx=0).stop 2024-09-19T06:10:23.812 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.253+0000 7fed0c83f640 1 -- 172.21.15.53:0/2858815569 shutdown_connections 2024-09-19T06:10:23.813 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.253+0000 7fed0c83f640 1 --2- 172.21.15.53:0/2858815569 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fecf00391e0 0x7fecf003b6a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.813 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.253+0000 7fed0c83f640 1 --2- 172.21.15.53:0/2858815569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed08106ac0 0x7fed0819aa00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.813 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.253+0000 7fed0c83f640 1 -- 172.21.15.53:0/2858815569 >> 172.21.15.53:0/2858815569 conn(0x7fed08102330 msgr2=0x7fed08102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:23.813 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.253+0000 7fed0c83f640 1 -- 172.21.15.53:0/2858815569 shutdown_connections 2024-09-19T06:10:23.813 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:23.253+0000 7fed0c83f640 1 -- 172.21.15.53:0/2858815569 wait complete. 2024-09-19T06:10:23.813 INFO:teuthology.orchestra.run.smithi053.stdout:Enabling the dashboard module... 2024-09-19T06:10:24.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:24 smithi053 ceph-mon[27968]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:10:24.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:24 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/2858815569' entity='client.admin' 2024-09-19T06:10:25.407 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:25 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/2089251796' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-19T06:10:25.715 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.190+0000 7fd8f23e4640 1 Processor -- start 2024-09-19T06:10:25.715 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.190+0000 7fd8f23e4640 1 -- start start 2024-09-19T06:10:25.715 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.190+0000 7fd8f23e4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8ec107590 0x7fd8ec109980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:25.715 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.190+0000 7fd8f23e4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec07a6b0 con 0x7fd8ec107590 2024-09-19T06:10:25.716 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.191+0000 7fd8ebfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8ec107590 0x7fd8ec109980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:25.716 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.191+0000 7fd8ebfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8ec107590 0x7fd8ec109980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44486/0 (socket says 172.21.15.53:44486) 2024-09-19T06:10:25.716 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.191+0000 7fd8ebfff640 1 -- 172.21.15.53:0/658609339 learned_addr learned my addr 172.21.15.53:0/658609339 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:25.716 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.191+0000 7fd8ebfff640 1 -- 172.21.15.53:0/658609339 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ec109ec0 con 0x7fd8ec107590 2024-09-19T06:10:25.716 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.192+0000 7fd8ebfff640 1 --2- 172.21.15.53:0/658609339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8ec107590 0x7fd8ec109980 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009820 tx=0x7fd8dc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac458bede2b7fab2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:25.716 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.192+0000 7fd8eaffd640 1 -- 172.21.15.53:0/658609339 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fd8dc037070 con 0x7fd8ec107590 2024-09-19T06:10:25.716 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.192+0000 7fd8eaffd640 1 -- 172.21.15.53:0/658609339 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8dc02ad40 con 0x7fd8ec107590 2024-09-19T06:10:25.716 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.192+0000 7fd8eaffd640 1 -- 172.21.15.53:0/658609339 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fd8dc030620 con 0x7fd8ec107590 2024-09-19T06:10:25.716 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.193+0000 7fd8f23e4640 1 -- 172.21.15.53:0/658609339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8ec107590 msgr2=0x7fd8ec109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:25.716 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.193+0000 7fd8f23e4640 1 --2- 172.21.15.53:0/658609339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8ec107590 0x7fd8ec109980 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009820 tx=0x7fd8dc02a270 comp rx=0 tx=0).stop 2024-09-19T06:10:25.716 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.193+0000 7fd8f23e4640 1 -- 172.21.15.53:0/658609339 shutdown_connections 2024-09-19T06:10:25.716 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.193+0000 7fd8f23e4640 1 --2- 172.21.15.53:0/658609339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8ec107590 0x7fd8ec109980 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:25.716 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.193+0000 7fd8f23e4640 1 -- 172.21.15.53:0/658609339 >> 172.21.15.53:0/658609339 conn(0x7fd8ec101190 msgr2=0x7fd8ec1035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:25.717 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.193+0000 7fd8f23e4640 1 -- 172.21.15.53:0/658609339 shutdown_connections 2024-09-19T06:10:25.717 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.193+0000 7fd8f23e4640 1 -- 172.21.15.53:0/658609339 wait complete. 2024-09-19T06:10:25.717 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.194+0000 7fd8f23e4640 1 Processor -- start 2024-09-19T06:10:25.717 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.194+0000 7fd8f23e4640 1 -- start start 2024-09-19T06:10:25.717 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.194+0000 7fd8f23e4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8ec107590 0x7fd8ec19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:25.717 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.194+0000 7fd8f23e4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec10ae80 con 0x7fd8ec107590 2024-09-19T06:10:25.717 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.194+0000 7fd8ebfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8ec107590 0x7fd8ec19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:25.717 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.194+0000 7fd8ebfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8ec107590 0x7fd8ec19aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44500/0 (socket says 172.21.15.53:44500) 2024-09-19T06:10:25.718 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.194+0000 7fd8ebfff640 1 -- 172.21.15.53:0/2089251796 learned_addr learned my addr 172.21.15.53:0/2089251796 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:25.718 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.195+0000 7fd8ebfff640 1 -- 172.21.15.53:0/2089251796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ec19af80 con 0x7fd8ec107590 2024-09-19T06:10:25.718 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.195+0000 7fd8ebfff640 1 --2- 172.21.15.53:0/2089251796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8ec107590 0x7fd8ec19aa40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc000c00 tx=0x7fd8dc030fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:25.718 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.195+0000 7fd8e97fa640 1 -- 172.21.15.53:0/2089251796 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fd8dc032a00 con 0x7fd8ec107590 2024-09-19T06:10:25.718 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.195+0000 7fd8e97fa640 1 -- 172.21.15.53:0/2089251796 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8dc032ba0 con 0x7fd8ec107590 2024-09-19T06:10:25.718 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.195+0000 7fd8e97fa640 1 -- 172.21.15.53:0/2089251796 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fd8dc03d5d0 con 0x7fd8ec107590 2024-09-19T06:10:25.718 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.195+0000 7fd8f23e4640 1 -- 172.21.15.53:0/2089251796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8ec19b210 con 0x7fd8ec107590 2024-09-19T06:10:25.718 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.195+0000 7fd8f23e4640 1 -- 172.21.15.53:0/2089251796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8ec19df00 con 0x7fd8ec107590 2024-09-19T06:10:25.718 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.196+0000 7fd8e97fa640 1 -- 172.21.15.53:0/2089251796 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) ==== 46476+0+0 (secure 0 0 0) 0x7fd8dc03d880 con 0x7fd8ec107590 2024-09-19T06:10:25.718 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.196+0000 7fd8e97fa640 1 --2- 172.21.15.53:0/2089251796 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fd8d4039230 0x7fd8d403b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:25.719 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.196+0000 7fd8e97fa640 1 -- 172.21.15.53:0/2089251796 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd8dc06bdf0 con 0x7fd8ec107590 2024-09-19T06:10:25.719 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.196+0000 7fd8f23e4640 1 -- 172.21.15.53:0/2089251796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8ec1079f0 con 0x7fd8ec107590 2024-09-19T06:10:25.719 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.196+0000 7fd8eb7fe640 1 --2- 172.21.15.53:0/2089251796 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fd8d4039230 0x7fd8d403b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:25.719 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.197+0000 7fd8eb7fe640 1 --2- 172.21.15.53:0/2089251796 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fd8d4039230 0x7fd8d403b6f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd8e00098c0 tx=0x7fd8e0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:25.719 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.201+0000 7fd8e97fa640 1 -- 172.21.15.53:0/2089251796 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd8dc0454d0 con 0x7fd8ec107590 2024-09-19T06:10:25.719 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:24.426+0000 7fd8f23e4640 1 -- 172.21.15.53:0/2089251796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fd8ec19e4d0 con 0x7fd8ec107590 2024-09-19T06:10:25.719 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:25.104+0000 7fd8e97fa640 1 -- 172.21.15.53:0/2089251796 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fd8dc045730 con 0x7fd8ec107590 2024-09-19T06:10:25.719 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:25.107+0000 7fd8e97fa640 1 -- 172.21.15.53:0/2089251796 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mgrmap(e 9) ==== 46489+0+0 (secure 0 0 0) 0x7fd8dc046360 con 0x7fd8ec107590 2024-09-19T06:10:25.719 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:25.107+0000 7fd8f23e4640 1 -- 172.21.15.53:0/2089251796 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fd8d4039230 msgr2=0x7fd8d403b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:25.719 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:25.107+0000 7fd8f23e4640 1 --2- 172.21.15.53:0/2089251796 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fd8d4039230 0x7fd8d403b6f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd8e00098c0 tx=0x7fd8e0006db0 comp rx=0 tx=0).stop 2024-09-19T06:10:25.719 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:25.107+0000 7fd8f23e4640 1 -- 172.21.15.53:0/2089251796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8ec107590 msgr2=0x7fd8ec19aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:25.719 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:25.107+0000 7fd8f23e4640 1 --2- 172.21.15.53:0/2089251796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8ec107590 0x7fd8ec19aa40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc000c00 tx=0x7fd8dc030fc0 comp rx=0 tx=0).stop 2024-09-19T06:10:25.719 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:25.108+0000 7fd8f23e4640 1 -- 172.21.15.53:0/2089251796 shutdown_connections 2024-09-19T06:10:25.720 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:25.108+0000 7fd8f23e4640 1 --2- 172.21.15.53:0/2089251796 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fd8d4039230 0x7fd8d403b6f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:25.720 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:25.108+0000 7fd8f23e4640 1 --2- 172.21.15.53:0/2089251796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8ec107590 0x7fd8ec19aa40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:25.720 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:25.108+0000 7fd8f23e4640 1 -- 172.21.15.53:0/2089251796 >> 172.21.15.53:0/2089251796 conn(0x7fd8ec101190 msgr2=0x7fd8ec101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:25.720 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:25.108+0000 7fd8f23e4640 1 -- 172.21.15.53:0/2089251796 shutdown_connections 2024-09-19T06:10:25.720 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:25.108+0000 7fd8f23e4640 1 -- 172.21.15.53:0/2089251796 wait complete. 2024-09-19T06:10:26.225 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:26 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/2089251796' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-19T06:10:26.225 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:26 smithi053 ceph-mon[27968]: mgrmap e9: smithi053.trakvg(active, since 26s) 2024-09-19T06:10:26.997 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout { 2024-09-19T06:10:26.997 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-19T06:10:26.997 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T06:10:26.997 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "active_name": "smithi053.trakvg", 2024-09-19T06:10:26.997 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T06:10:26.998 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout } 2024-09-19T06:10:26.998 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.157+0000 7f5fea84c640 1 Processor -- start 2024-09-19T06:10:26.998 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.157+0000 7f5fea84c640 1 -- start start 2024-09-19T06:10:26.998 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.157+0000 7f5fea84c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe4106b20 0x7f5fe4106f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:26.998 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.157+0000 7f5fea84c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe41074f0 con 0x7f5fe4106b20 2024-09-19T06:10:26.998 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.158+0000 7f5fe3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe4106b20 0x7f5fe4106f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:26.998 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.158+0000 7f5fe3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe4106b20 0x7f5fe4106f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44526/0 (socket says 172.21.15.53:44526) 2024-09-19T06:10:26.998 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.158+0000 7f5fe3fff640 1 -- 172.21.15.53:0/761987894 learned_addr learned my addr 172.21.15.53:0/761987894 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:26.998 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.158+0000 7f5fe3fff640 1 -- 172.21.15.53:0/761987894 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe4107c80 con 0x7f5fe4106b20 2024-09-19T06:10:26.998 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.159+0000 7f5fe3fff640 1 --2- 172.21.15.53:0/761987894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe4106b20 0x7f5fe4106f20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009820 tx=0x7f5fd402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be7ffcbe8f579ccf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:26.998 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.159+0000 7f5fe2ffd640 1 -- 172.21.15.53:0/761987894 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f5fd4037070 con 0x7f5fe4106b20 2024-09-19T06:10:26.998 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.159+0000 7f5fe2ffd640 1 -- 172.21.15.53:0/761987894 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fd402ad40 con 0x7f5fe4106b20 2024-09-19T06:10:26.998 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.159+0000 7f5fe2ffd640 1 -- 172.21.15.53:0/761987894 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f5fd4030620 con 0x7f5fe4106b20 2024-09-19T06:10:26.999 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.160+0000 7f5fea84c640 1 -- 172.21.15.53:0/761987894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe4106b20 msgr2=0x7f5fe4106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:26.999 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.160+0000 7f5fea84c640 1 --2- 172.21.15.53:0/761987894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe4106b20 0x7f5fe4106f20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009820 tx=0x7f5fd402a270 comp rx=0 tx=0).stop 2024-09-19T06:10:26.999 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.160+0000 7f5fea84c640 1 -- 172.21.15.53:0/761987894 shutdown_connections 2024-09-19T06:10:26.999 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.160+0000 7f5fea84c640 1 --2- 172.21.15.53:0/761987894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe4106b20 0x7f5fe4106f20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:26.999 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.160+0000 7f5fea84c640 1 -- 172.21.15.53:0/761987894 >> 172.21.15.53:0/761987894 conn(0x7f5fe41022f0 msgr2=0x7f5fe4104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:26.999 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.160+0000 7f5fea84c640 1 -- 172.21.15.53:0/761987894 shutdown_connections 2024-09-19T06:10:26.999 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.160+0000 7f5fea84c640 1 -- 172.21.15.53:0/761987894 wait complete. 2024-09-19T06:10:26.999 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.161+0000 7f5fea84c640 1 Processor -- start 2024-09-19T06:10:26.999 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.161+0000 7f5fea84c640 1 -- start start 2024-09-19T06:10:26.999 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.161+0000 7f5fea84c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe4106b20 0x7f5fe419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:26.999 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.161+0000 7f5fea84c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe410b030 con 0x7f5fe4106b20 2024-09-19T06:10:26.999 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.161+0000 7f5fe3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe4106b20 0x7f5fe419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:26.999 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.161+0000 7f5fe3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe4106b20 0x7f5fe419aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44532/0 (socket says 172.21.15.53:44532) 2024-09-19T06:10:26.999 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.161+0000 7f5fe3fff640 1 -- 172.21.15.53:0/808652614 learned_addr learned my addr 172.21.15.53:0/808652614 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:27.000 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.162+0000 7f5fe3fff640 1 -- 172.21.15.53:0/808652614 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe419afb0 con 0x7f5fe4106b20 2024-09-19T06:10:27.000 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.162+0000 7f5fe3fff640 1 --2- 172.21.15.53:0/808652614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe4106b20 0x7f5fe419aa70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4006f10 tx=0x7f5fd4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:27.000 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.162+0000 7f5fe17fa640 1 -- 172.21.15.53:0/808652614 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f5fd4032a40 con 0x7f5fe4106b20 2024-09-19T06:10:27.000 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.162+0000 7f5fe17fa640 1 -- 172.21.15.53:0/808652614 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fd4032be0 con 0x7f5fe4106b20 2024-09-19T06:10:27.000 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.162+0000 7f5fea84c640 1 -- 172.21.15.53:0/808652614 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe419b240 con 0x7f5fe4106b20 2024-09-19T06:10:27.000 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.162+0000 7f5fea84c640 1 -- 172.21.15.53:0/808652614 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe419df30 con 0x7f5fe4106b20 2024-09-19T06:10:27.000 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.162+0000 7f5fe17fa640 1 -- 172.21.15.53:0/808652614 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f5fd403d590 con 0x7f5fe4106b20 2024-09-19T06:10:27.000 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.163+0000 7f5fe17fa640 1 -- 172.21.15.53:0/808652614 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 9) ==== 46489+0+0 (secure 0 0 0) 0x7f5fd403d880 con 0x7f5fe4106b20 2024-09-19T06:10:27.000 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.163+0000 7f5fea84c640 1 -- 172.21.15.53:0/808652614 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fe410b1b0 con 0x7f5fe4106b20 2024-09-19T06:10:27.000 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.163+0000 7f5fe17fa640 1 --2- 172.21.15.53:0/808652614 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f5fcc039290 0x7f5fcc03b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:27.000 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.163+0000 7f5fe17fa640 1 -- 172.21.15.53:0/808652614 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5fd406cc90 con 0x7f5fe4106b20 2024-09-19T06:10:27.000 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.164+0000 7f5fe37fe640 1 -- 172.21.15.53:0/808652614 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f5fcc039290 msgr2=0x7f5fcc03b750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/681468213 2024-09-19T06:10:27.000 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.164+0000 7f5fe37fe640 1 --2- 172.21.15.53:0/808652614 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f5fcc039290 0x7f5fcc03b750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T06:10:27.001 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.168+0000 7f5fe17fa640 1 -- 172.21.15.53:0/808652614 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5fd406c9f0 con 0x7f5fe4106b20 2024-09-19T06:10:27.001 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.365+0000 7f5fe37fe640 1 -- 172.21.15.53:0/808652614 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f5fcc039290 msgr2=0x7f5fcc03b750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/681468213 2024-09-19T06:10:27.001 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.365+0000 7f5fe37fe640 1 --2- 172.21.15.53:0/808652614 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f5fcc039290 0x7f5fcc03b750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T06:10:27.001 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.391+0000 7f5fea84c640 1 -- 172.21.15.53:0/808652614 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f5fe419e100 con 0x7f5fe4106b20 2024-09-19T06:10:27.001 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.392+0000 7f5fe17fa640 1 -- 172.21.15.53:0/808652614 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+103 (secure 0 0 0) 0x7f5fd4046360 con 0x7f5fe4106b20 2024-09-19T06:10:27.001 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.395+0000 7f5fea84c640 1 -- 172.21.15.53:0/808652614 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f5fcc039290 msgr2=0x7f5fcc03b750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T06:10:27.001 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.395+0000 7f5fea84c640 1 --2- 172.21.15.53:0/808652614 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f5fcc039290 0x7f5fcc03b750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.001 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.395+0000 7f5fea84c640 1 -- 172.21.15.53:0/808652614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe4106b20 msgr2=0x7f5fe419aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:27.001 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.395+0000 7f5fea84c640 1 --2- 172.21.15.53:0/808652614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe4106b20 0x7f5fe419aa70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4006f10 tx=0x7f5fd4032670 comp rx=0 tx=0).stop 2024-09-19T06:10:27.001 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.395+0000 7f5fea84c640 1 -- 172.21.15.53:0/808652614 shutdown_connections 2024-09-19T06:10:27.001 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.395+0000 7f5fea84c640 1 --2- 172.21.15.53:0/808652614 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7f5fcc039290 0x7f5fcc03b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.001 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.395+0000 7f5fea84c640 1 --2- 172.21.15.53:0/808652614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe4106b20 0x7f5fe419aa70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.001 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.395+0000 7f5fea84c640 1 -- 172.21.15.53:0/808652614 >> 172.21.15.53:0/808652614 conn(0x7f5fe41022f0 msgr2=0x7f5fe4102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:27.002 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.395+0000 7f5fea84c640 1 -- 172.21.15.53:0/808652614 shutdown_connections 2024-09-19T06:10:27.002 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:26.395+0000 7f5fea84c640 1 -- 172.21.15.53:0/808652614 wait complete. 2024-09-19T06:10:27.002 INFO:teuthology.orchestra.run.smithi053.stdout:Waiting for the mgr to restart... 2024-09-19T06:10:27.002 INFO:teuthology.orchestra.run.smithi053.stdout:Waiting for mgr epoch 9... 2024-09-19T06:10:27.324 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:27 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/808652614' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T06:10:30.465 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:30 smithi053 ceph-mon[27968]: Active manager daemon smithi053.trakvg restarted 2024-09-19T06:10:30.465 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:30 smithi053 ceph-mon[27968]: Activating manager daemon smithi053.trakvg 2024-09-19T06:10:30.465 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:30 smithi053 ceph-mon[27968]: osdmap e3: 0 total, 0 up, 0 in 2024-09-19T06:10:30.465 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:30 smithi053 ceph-mon[27968]: mgrmap e10: smithi053.trakvg(active, starting, since 0.0035015s) 2024-09-19T06:10:30.465 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:30 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata", "id": "smithi053"}]: dispatch 2024-09-19T06:10:30.465 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:30 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mgr metadata", "who": "smithi053.trakvg", "id": "smithi053.trakvg"}]: dispatch 2024-09-19T06:10:30.465 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:30 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T06:10:30.466 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:30 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T06:10:30.466 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:30 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T06:10:30.466 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:30 smithi053 ceph-mon[27968]: Manager daemon smithi053.trakvg is now available 2024-09-19T06:10:30.466 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:30 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:10:30.466 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:30 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi053.trakvg/mirror_snapshot_schedule"}]: dispatch 2024-09-19T06:10:30.466 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:30 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi053.trakvg/trash_purge_schedule"}]: dispatch 2024-09-19T06:10:32.141 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout { 2024-09-19T06:10:32.141 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-19T06:10:32.141 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T06:10:32.141 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout } 2024-09-19T06:10:32.142 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.404+0000 7fb7b2a08640 1 Processor -- start 2024-09-19T06:10:32.142 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.404+0000 7fb7b2a08640 1 -- start start 2024-09-19T06:10:32.142 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.404+0000 7fb7b2a08640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7ac1075a0 0x7fb7ac109990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.142 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.404+0000 7fb7b2a08640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ac07a6b0 con 0x7fb7ac1075a0 2024-09-19T06:10:32.142 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.405+0000 7fb7abfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7ac1075a0 0x7fb7ac109990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.142 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.405+0000 7fb7abfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7ac1075a0 0x7fb7ac109990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44542/0 (socket says 172.21.15.53:44542) 2024-09-19T06:10:32.142 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.405+0000 7fb7abfff640 1 -- 172.21.15.53:0/3997083888 learned_addr learned my addr 172.21.15.53:0/3997083888 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:32.142 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.405+0000 7fb7abfff640 1 -- 172.21.15.53:0/3997083888 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7ac109ed0 con 0x7fb7ac1075a0 2024-09-19T06:10:32.142 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.406+0000 7fb7abfff640 1 --2- 172.21.15.53:0/3997083888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7ac1075a0 0x7fb7ac109990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb79c009820 tx=0x7fb79c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401b7e089974e28 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:32.142 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.406+0000 7fb7aaffd640 1 -- 172.21.15.53:0/3997083888 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fb79c037070 con 0x7fb7ac1075a0 2024-09-19T06:10:32.143 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.406+0000 7fb7aaffd640 1 -- 172.21.15.53:0/3997083888 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb79c02ad40 con 0x7fb7ac1075a0 2024-09-19T06:10:32.143 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.406+0000 7fb7aaffd640 1 -- 172.21.15.53:0/3997083888 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fb79c030620 con 0x7fb7ac1075a0 2024-09-19T06:10:32.143 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.406+0000 7fb7b2a08640 1 -- 172.21.15.53:0/3997083888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7ac1075a0 msgr2=0x7fb7ac109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:32.143 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.406+0000 7fb7b2a08640 1 --2- 172.21.15.53:0/3997083888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7ac1075a0 0x7fb7ac109990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb79c009820 tx=0x7fb79c02a270 comp rx=0 tx=0).stop 2024-09-19T06:10:32.143 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.406+0000 7fb7b2a08640 1 -- 172.21.15.53:0/3997083888 shutdown_connections 2024-09-19T06:10:32.143 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.406+0000 7fb7b2a08640 1 --2- 172.21.15.53:0/3997083888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7ac1075a0 0x7fb7ac109990 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.143 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.407+0000 7fb7b2a08640 1 -- 172.21.15.53:0/3997083888 >> 172.21.15.53:0/3997083888 conn(0x7fb7ac1011a0 msgr2=0x7fb7ac1035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:32.143 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.407+0000 7fb7b2a08640 1 -- 172.21.15.53:0/3997083888 shutdown_connections 2024-09-19T06:10:32.143 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.407+0000 7fb7b2a08640 1 -- 172.21.15.53:0/3997083888 wait complete. 2024-09-19T06:10:32.143 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.407+0000 7fb7b2a08640 1 Processor -- start 2024-09-19T06:10:32.143 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.407+0000 7fb7b2a08640 1 -- start start 2024-09-19T06:10:32.143 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.407+0000 7fb7b2a08640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7ac1075a0 0x7fb7ac19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.143 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.408+0000 7fb7b2a08640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ac10ae90 con 0x7fb7ac1075a0 2024-09-19T06:10:32.143 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.408+0000 7fb7abfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7ac1075a0 0x7fb7ac19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.144 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.408+0000 7fb7abfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7ac1075a0 0x7fb7ac19aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44554/0 (socket says 172.21.15.53:44554) 2024-09-19T06:10:32.144 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.408+0000 7fb7abfff640 1 -- 172.21.15.53:0/1596823428 learned_addr learned my addr 172.21.15.53:0/1596823428 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:32.144 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.408+0000 7fb7abfff640 1 -- 172.21.15.53:0/1596823428 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7ac19afb0 con 0x7fb7ac1075a0 2024-09-19T06:10:32.144 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.408+0000 7fb7abfff640 1 --2- 172.21.15.53:0/1596823428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7ac1075a0 0x7fb7ac19aa70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb79c000c00 tx=0x7fb79c030fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:32.144 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.408+0000 7fb7a97fa640 1 -- 172.21.15.53:0/1596823428 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fb79c032a00 con 0x7fb7ac1075a0 2024-09-19T06:10:32.144 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.408+0000 7fb7b2a08640 1 -- 172.21.15.53:0/1596823428 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7ac19b240 con 0x7fb7ac1075a0 2024-09-19T06:10:32.144 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.408+0000 7fb7a97fa640 1 -- 172.21.15.53:0/1596823428 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb79c032ba0 con 0x7fb7ac1075a0 2024-09-19T06:10:32.144 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.408+0000 7fb7a97fa640 1 -- 172.21.15.53:0/1596823428 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fb79c03d5d0 con 0x7fb7ac1075a0 2024-09-19T06:10:32.144 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.408+0000 7fb7b2a08640 1 -- 172.21.15.53:0/1596823428 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7ac19df30 con 0x7fb7ac1075a0 2024-09-19T06:10:32.144 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.409+0000 7fb7a97fa640 1 -- 172.21.15.53:0/1596823428 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 9) ==== 46489+0+0 (secure 0 0 0) 0x7fb79c03d8d0 con 0x7fb7ac1075a0 2024-09-19T06:10:32.144 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.409+0000 7fb7a97fa640 1 --2- 172.21.15.53:0/1596823428 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fb794039230 0x7fb79403b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.144 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.409+0000 7fb7b2a08640 1 -- 172.21.15.53:0/1596823428 --> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb7ac107a00 con 0x7fb794039230 2024-09-19T06:10:32.144 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.409+0000 7fb7a97fa640 1 -- 172.21.15.53:0/1596823428 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb79c06beb0 con 0x7fb7ac1075a0 2024-09-19T06:10:32.144 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.409+0000 7fb7ab7fe640 1 -- 172.21.15.53:0/1596823428 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fb794039230 msgr2=0x7fb79403b6f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/681468213 2024-09-19T06:10:32.145 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.410+0000 7fb7ab7fe640 1 --2- 172.21.15.53:0/1596823428 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fb794039230 0x7fb79403b6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T06:10:32.145 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.611+0000 7fb7ab7fe640 1 -- 172.21.15.53:0/1596823428 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fb794039230 msgr2=0x7fb79403b6f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/681468213 2024-09-19T06:10:32.145 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:27.611+0000 7fb7ab7fe640 1 --2- 172.21.15.53:0/1596823428 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fb794039230 0x7fb79403b6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T06:10:32.145 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:28.012+0000 7fb7ab7fe640 1 -- 172.21.15.53:0/1596823428 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fb794039230 msgr2=0x7fb79403b6f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/681468213 2024-09-19T06:10:32.145 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:28.012+0000 7fb7ab7fe640 1 --2- 172.21.15.53:0/1596823428 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fb794039230 0x7fb79403b6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T06:10:32.145 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:28.813+0000 7fb7ab7fe640 1 -- 172.21.15.53:0/1596823428 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fb794039230 msgr2=0x7fb79403b6f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/681468213 2024-09-19T06:10:32.145 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:28.813+0000 7fb7ab7fe640 1 --2- 172.21.15.53:0/1596823428 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fb794039230 0x7fb79403b6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T06:10:32.145 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:30.210+0000 7fb7a97fa640 1 -- 172.21.15.53:0/1596823428 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mgrmap(e 10) ==== 46256+0+0 (secure 0 0 0) 0x7fb79c06acb0 con 0x7fb7ac1075a0 2024-09-19T06:10:32.145 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:30.210+0000 7fb7a97fa640 1 -- 172.21.15.53:0/1596823428 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fb794039230 msgr2=0x7fb79403b6f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T06:10:32.145 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:30.210+0000 7fb7a97fa640 1 --2- 172.21.15.53:0/1596823428 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fb794039230 0x7fb79403b6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.145 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:31.211+0000 7fb7a97fa640 1 -- 172.21.15.53:0/1596823428 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 11) ==== 46383+0+0 (secure 0 0 0) 0x7fb79c02fd60 con 0x7fb7ac1075a0 2024-09-19T06:10:32.145 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:31.211+0000 7fb7a97fa640 1 --2- 172.21.15.53:0/1596823428 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fb79403cb00 0x7fb79403eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.145 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:31.211+0000 7fb7a97fa640 1 -- 172.21.15.53:0/1596823428 --> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb7ac107a00 con 0x7fb79403cb00 2024-09-19T06:10:32.146 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:31.212+0000 7fb7ab7fe640 1 --2- 172.21.15.53:0/1596823428 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fb79403cb00 0x7fb79403eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.146 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:31.213+0000 7fb7ab7fe640 1 --2- 172.21.15.53:0/1596823428 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fb79403cb00 0x7fb79403eef0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0003980 tx=0x7fb7a0006e10 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:32.146 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:31.213+0000 7fb7a97fa640 1 -- 172.21.15.53:0/1596823428 <== mgr.14162 v2:172.21.15.53:6800/3684090301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fb7ac107a00 con 0x7fb79403cb00 2024-09-19T06:10:32.146 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:31.220+0000 7fb7b2a08640 1 -- 172.21.15.53:0/1596823428 --> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fb7ac102470 con 0x7fb79403cb00 2024-09-19T06:10:32.146 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:31.220+0000 7fb7a97fa640 1 -- 172.21.15.53:0/1596823428 <== mgr.14162 v2:172.21.15.53:6800/3684090301 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fb7ac102470 con 0x7fb79403cb00 2024-09-19T06:10:32.146 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:31.221+0000 7fb7b2a08640 1 -- 172.21.15.53:0/1596823428 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fb79403cb00 msgr2=0x7fb79403eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:32.146 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:31.221+0000 7fb7b2a08640 1 --2- 172.21.15.53:0/1596823428 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fb79403cb00 0x7fb79403eef0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0003980 tx=0x7fb7a0006e10 comp rx=0 tx=0).stop 2024-09-19T06:10:32.146 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:31.221+0000 7fb7b2a08640 1 -- 172.21.15.53:0/1596823428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7ac1075a0 msgr2=0x7fb7ac19aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:32.146 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:31.221+0000 7fb7b2a08640 1 --2- 172.21.15.53:0/1596823428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7ac1075a0 0x7fb7ac19aa70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb79c000c00 tx=0x7fb79c030fc0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.146 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:31.221+0000 7fb7b2a08640 1 -- 172.21.15.53:0/1596823428 shutdown_connections 2024-09-19T06:10:32.146 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:31.221+0000 7fb7b2a08640 1 --2- 172.21.15.53:0/1596823428 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fb79403cb00 0x7fb79403eef0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.146 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:31.221+0000 7fb7b2a08640 1 --2- 172.21.15.53:0/1596823428 >> [v2:172.21.15.53:6800/681468213,v1:172.21.15.53:6801/681468213] conn(0x7fb794039230 0x7fb79403b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.146 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:31.221+0000 7fb7b2a08640 1 --2- 172.21.15.53:0/1596823428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7ac1075a0 0x7fb7ac19aa70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.146 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:31.221+0000 7fb7b2a08640 1 -- 172.21.15.53:0/1596823428 >> 172.21.15.53:0/1596823428 conn(0x7fb7ac1011a0 msgr2=0x7fb7ac101d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:32.147 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:31.221+0000 7fb7b2a08640 1 -- 172.21.15.53:0/1596823428 shutdown_connections 2024-09-19T06:10:32.147 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:31.221+0000 7fb7b2a08640 1 -- 172.21.15.53:0/1596823428 wait complete. 2024-09-19T06:10:32.147 INFO:teuthology.orchestra.run.smithi053.stdout:mgr epoch 9 is available 2024-09-19T06:10:32.147 INFO:teuthology.orchestra.run.smithi053.stdout:Generating a dashboard self-signed certificate... 2024-09-19T06:10:32.382 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:32 smithi053 ceph-mon[27968]: mgrmap e11: smithi053.trakvg(active, since 1.00638s) 2024-09-19T06:10:32.382 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:32 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:32.382 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:32 smithi053 ceph-mon[27968]: [19/Sep/2024:06:10:32] ENGINE Bus STARTING 2024-09-19T06:10:32.382 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:32 smithi053 ceph-mon[27968]: [19/Sep/2024:06:10:32] ENGINE Serving on https://172.21.15.53:7150 2024-09-19T06:10:32.382 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:32 smithi053 ceph-mon[27968]: [19/Sep/2024:06:10:32] ENGINE Error in HTTPServer.serve 2024-09-19T06:10:32.382 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Traceback (most recent call last): 2024-09-19T06:10:32.382 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T06:10:32.382 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: self._connections.run(self.expiration_interval) 2024-09-19T06:10:32.382 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T06:10:32.383 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: self._run(expiration_interval) 2024-09-19T06:10:32.383 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T06:10:32.383 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T06:10:32.383 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T06:10:32.383 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T06:10:32.383 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T06:10:32.383 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: s = self.context.wrap_socket( 2024-09-19T06:10:32.383 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T06:10:32.383 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: return self.sslsocket_class._create( 2024-09-19T06:10:32.383 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T06:10:32.383 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: self.do_handshake() 2024-09-19T06:10:32.383 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T06:10:32.383 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: self._sslobj.do_handshake() 2024-09-19T06:10:32.383 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T06:10:33.448 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-19T06:10:33.448 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.606+0000 7f47b54a4640 1 Processor -- start 2024-09-19T06:10:33.448 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.607+0000 7f47b54a4640 1 -- start start 2024-09-19T06:10:33.448 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.607+0000 7f47b54a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47b0107510 0x7f47b0109900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:33.448 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.607+0000 7f47b54a4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f47b007a730 con 0x7f47b0107510 2024-09-19T06:10:33.448 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.607+0000 7f47aeffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47b0107510 0x7f47b0109900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:33.448 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.607+0000 7f47aeffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47b0107510 0x7f47b0109900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45096/0 (socket says 172.21.15.53:45096) 2024-09-19T06:10:33.448 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.607+0000 7f47aeffd640 1 -- 172.21.15.53:0/1468328969 learned_addr learned my addr 172.21.15.53:0/1468328969 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:33.449 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.607+0000 7f47aeffd640 1 -- 172.21.15.53:0/1468328969 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b0109e40 con 0x7f47b0107510 2024-09-19T06:10:33.449 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.608+0000 7f47aeffd640 1 --2- 172.21.15.53:0/1468328969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47b0107510 0x7f47b0109900 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f47a0009820 tx=0x7f47a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5f10934ef91286 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:33.449 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.608+0000 7f47adffb640 1 -- 172.21.15.53:0/1468328969 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f47a0037070 con 0x7f47b0107510 2024-09-19T06:10:33.449 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.608+0000 7f47adffb640 1 -- 172.21.15.53:0/1468328969 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47a002ad40 con 0x7f47b0107510 2024-09-19T06:10:33.449 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.608+0000 7f47adffb640 1 -- 172.21.15.53:0/1468328969 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f47a0030620 con 0x7f47b0107510 2024-09-19T06:10:33.449 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.609+0000 7f47b54a4640 1 -- 172.21.15.53:0/1468328969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47b0107510 msgr2=0x7f47b0109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:33.449 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.609+0000 7f47b54a4640 1 --2- 172.21.15.53:0/1468328969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47b0107510 0x7f47b0109900 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f47a0009820 tx=0x7f47a002a270 comp rx=0 tx=0).stop 2024-09-19T06:10:33.449 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.609+0000 7f47b54a4640 1 -- 172.21.15.53:0/1468328969 shutdown_connections 2024-09-19T06:10:33.449 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.609+0000 7f47b54a4640 1 --2- 172.21.15.53:0/1468328969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47b0107510 0x7f47b0109900 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.450 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.609+0000 7f47b54a4640 1 -- 172.21.15.53:0/1468328969 >> 172.21.15.53:0/1468328969 conn(0x7f47b0101190 msgr2=0x7f47b01035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:33.450 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.609+0000 7f47b54a4640 1 -- 172.21.15.53:0/1468328969 shutdown_connections 2024-09-19T06:10:33.450 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.609+0000 7f47b54a4640 1 -- 172.21.15.53:0/1468328969 wait complete. 2024-09-19T06:10:33.450 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.610+0000 7f47b54a4640 1 Processor -- start 2024-09-19T06:10:33.450 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.610+0000 7f47b54a4640 1 -- start start 2024-09-19T06:10:33.450 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.610+0000 7f47b54a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47b0107510 0x7f47b019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:33.450 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.610+0000 7f47b54a4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f47b010aec0 con 0x7f47b0107510 2024-09-19T06:10:33.450 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.610+0000 7f47aeffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47b0107510 0x7f47b019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:33.450 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.610+0000 7f47aeffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47b0107510 0x7f47b019cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45104/0 (socket says 172.21.15.53:45104) 2024-09-19T06:10:33.450 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.610+0000 7f47aeffd640 1 -- 172.21.15.53:0/2525256147 learned_addr learned my addr 172.21.15.53:0/2525256147 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:33.450 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.611+0000 7f47aeffd640 1 -- 172.21.15.53:0/2525256147 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b019d1a0 con 0x7f47b0107510 2024-09-19T06:10:33.450 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.611+0000 7f47aeffd640 1 --2- 172.21.15.53:0/2525256147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47b0107510 0x7f47b019cc60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f47a0006f10 tx=0x7f47a0032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:33.450 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.611+0000 7f4797fff640 1 -- 172.21.15.53:0/2525256147 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f47a0032a70 con 0x7f47b0107510 2024-09-19T06:10:33.450 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.611+0000 7f47b54a4640 1 -- 172.21.15.53:0/2525256147 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47b019d430 con 0x7f47b0107510 2024-09-19T06:10:33.451 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.611+0000 7f4797fff640 1 -- 172.21.15.53:0/2525256147 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47a0032c10 con 0x7f47b0107510 2024-09-19T06:10:33.451 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.611+0000 7f4797fff640 1 -- 172.21.15.53:0/2525256147 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f47a003d620 con 0x7f47b0107510 2024-09-19T06:10:33.451 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.611+0000 7f47b54a4640 1 -- 172.21.15.53:0/2525256147 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47b01a0120 con 0x7f47b0107510 2024-09-19T06:10:33.451 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.612+0000 7f4797fff640 1 -- 172.21.15.53:0/2525256147 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 12) ==== 46489+0+0 (secure 0 0 0) 0x7f47a003d920 con 0x7f47b0107510 2024-09-19T06:10:33.451 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.612+0000 7f4797fff640 1 --2- 172.21.15.53:0/2525256147 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f4798039230 0x7f479803b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:33.451 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.612+0000 7f4797fff640 1 -- 172.21.15.53:0/2525256147 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f47a006d170 con 0x7f47b0107510 2024-09-19T06:10:33.451 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.612+0000 7f47b54a4640 1 -- 172.21.15.53:0/2525256147 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47b0107970 con 0x7f47b0107510 2024-09-19T06:10:33.451 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.612+0000 7f47ae7fc640 1 --2- 172.21.15.53:0/2525256147 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f4798039230 0x7f479803b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:33.451 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.613+0000 7f47ae7fc640 1 --2- 172.21.15.53:0/2525256147 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f4798039230 0x7f479803b6f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f47a40098c0 tx=0x7f47a4006db0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:33.451 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.617+0000 7f4797fff640 1 -- 172.21.15.53:0/2525256147 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f47a00454d0 con 0x7f47b0107510 2024-09-19T06:10:33.451 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.816+0000 7f47b54a4640 1 -- 172.21.15.53:0/2525256147 --> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f47b0102390 con 0x7f4798039230 2024-09-19T06:10:33.451 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.872+0000 7f4797fff640 1 -- 172.21.15.53:0/2525256147 <== mgr.14162 v2:172.21.15.53:6800/3684090301 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f47b0102390 con 0x7f4798039230 2024-09-19T06:10:33.451 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.874+0000 7f47b54a4640 1 -- 172.21.15.53:0/2525256147 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f4798039230 msgr2=0x7f479803b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:33.451 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.874+0000 7f47b54a4640 1 --2- 172.21.15.53:0/2525256147 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f4798039230 0x7f479803b6f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f47a40098c0 tx=0x7f47a4006db0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.451 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.874+0000 7f47b54a4640 1 -- 172.21.15.53:0/2525256147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47b0107510 msgr2=0x7f47b019cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:33.452 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.874+0000 7f47b54a4640 1 --2- 172.21.15.53:0/2525256147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47b0107510 0x7f47b019cc60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f47a0006f10 tx=0x7f47a0032670 comp rx=0 tx=0).stop 2024-09-19T06:10:33.452 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.874+0000 7f47b54a4640 1 -- 172.21.15.53:0/2525256147 shutdown_connections 2024-09-19T06:10:33.452 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.874+0000 7f47b54a4640 1 --2- 172.21.15.53:0/2525256147 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f4798039230 0x7f479803b6f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.452 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.874+0000 7f47b54a4640 1 --2- 172.21.15.53:0/2525256147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47b0107510 0x7f47b019cc60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.452 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.874+0000 7f47b54a4640 1 -- 172.21.15.53:0/2525256147 >> 172.21.15.53:0/2525256147 conn(0x7f47b0101190 msgr2=0x7f47b0101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:33.452 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.875+0000 7f47b54a4640 1 -- 172.21.15.53:0/2525256147 shutdown_connections 2024-09-19T06:10:33.452 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:32.875+0000 7f47b54a4640 1 -- 172.21.15.53:0/2525256147 wait complete. 2024-09-19T06:10:33.452 INFO:teuthology.orchestra.run.smithi053.stdout:Creating initial admin user... 2024-09-19T06:10:33.541 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:33 smithi053 ceph-mon[27968]: [19/Sep/2024:06:10:32] ENGINE Serving on http://172.21.15.53:8765 2024-09-19T06:10:33.541 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:33 smithi053 ceph-mon[27968]: [19/Sep/2024:06:10:32] ENGINE Bus STARTED 2024-09-19T06:10:33.541 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:33 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:33.541 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:33 smithi053 ceph-mon[27968]: mgrmap e12: smithi053.trakvg(active, since 2s) 2024-09-19T06:10:33.541 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:33 smithi053 ceph-mon[27968]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:10:33.541 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:33 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:33.541 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:33 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:35.671 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$kQEkNKL5NlfimZedFvN3ke2zWe2sZmf97Vi4o0Pa6vUyDJEuAxmTq", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726726235, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-19T06:10:35.671 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.598+0000 7fecac6d4640 1 Processor -- start 2024-09-19T06:10:35.672 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.598+0000 7fecac6d4640 1 -- start start 2024-09-19T06:10:35.672 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.598+0000 7fecac6d4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feca4106ac0 0x7feca4106ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:35.672 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.598+0000 7fecac6d4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7feca4107400 con 0x7feca4106ac0 2024-09-19T06:10:35.672 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.598+0000 7fecaa470640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feca4106ac0 0x7feca4106ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:35.672 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.598+0000 7fecaa470640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feca4106ac0 0x7feca4106ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45120/0 (socket says 172.21.15.53:45120) 2024-09-19T06:10:35.672 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.598+0000 7fecaa470640 1 -- 172.21.15.53:0/1023997104 learned_addr learned my addr 172.21.15.53:0/1023997104 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:35.672 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.599+0000 7fecaa470640 1 -- 172.21.15.53:0/1023997104 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca4107580 con 0x7feca4106ac0 2024-09-19T06:10:35.672 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.599+0000 7fecaa470640 1 --2- 172.21.15.53:0/1023997104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feca4106ac0 0x7feca4106ec0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7feca0009820 tx=0x7feca002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20aef831fb95b7a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:35.672 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.600+0000 7feca946e640 1 -- 172.21.15.53:0/1023997104 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7feca0037070 con 0x7feca4106ac0 2024-09-19T06:10:35.672 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.600+0000 7feca946e640 1 -- 172.21.15.53:0/1023997104 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feca002ad40 con 0x7feca4106ac0 2024-09-19T06:10:35.672 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.600+0000 7feca946e640 1 -- 172.21.15.53:0/1023997104 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7feca0030620 con 0x7feca4106ac0 2024-09-19T06:10:35.672 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.600+0000 7fecac6d4640 1 -- 172.21.15.53:0/1023997104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feca4106ac0 msgr2=0x7feca4106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:35.673 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.600+0000 7fecac6d4640 1 --2- 172.21.15.53:0/1023997104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feca4106ac0 0x7feca4106ec0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7feca0009820 tx=0x7feca002a270 comp rx=0 tx=0).stop 2024-09-19T06:10:35.673 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.600+0000 7fecac6d4640 1 -- 172.21.15.53:0/1023997104 shutdown_connections 2024-09-19T06:10:35.673 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.600+0000 7fecac6d4640 1 --2- 172.21.15.53:0/1023997104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feca4106ac0 0x7feca4106ec0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:35.673 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.600+0000 7fecac6d4640 1 -- 172.21.15.53:0/1023997104 >> 172.21.15.53:0/1023997104 conn(0x7feca4102330 msgr2=0x7feca4104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:35.673 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.600+0000 7fecac6d4640 1 -- 172.21.15.53:0/1023997104 shutdown_connections 2024-09-19T06:10:35.673 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.600+0000 7fecac6d4640 1 -- 172.21.15.53:0/1023997104 wait complete. 2024-09-19T06:10:35.673 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.601+0000 7fecac6d4640 1 Processor -- start 2024-09-19T06:10:35.673 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.601+0000 7fecac6d4640 1 -- start start 2024-09-19T06:10:35.673 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.601+0000 7fecac6d4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feca4106ac0 0x7feca419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:35.673 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.601+0000 7fecac6d4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7feca410af30 con 0x7feca4106ac0 2024-09-19T06:10:35.673 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.601+0000 7fecaa470640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feca4106ac0 0x7feca419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:35.673 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.602+0000 7fecaa470640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feca4106ac0 0x7feca419aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45136/0 (socket says 172.21.15.53:45136) 2024-09-19T06:10:35.674 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.602+0000 7fecaa470640 1 -- 172.21.15.53:0/2397078035 learned_addr learned my addr 172.21.15.53:0/2397078035 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:35.674 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.602+0000 7fecaa470640 1 -- 172.21.15.53:0/2397078035 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca419af40 con 0x7feca4106ac0 2024-09-19T06:10:35.674 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.602+0000 7fecaa470640 1 --2- 172.21.15.53:0/2397078035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feca4106ac0 0x7feca419aa00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feca0006f10 tx=0x7feca0032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:35.674 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.602+0000 7fec9b7fe640 1 -- 172.21.15.53:0/2397078035 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7feca0032a70 con 0x7feca4106ac0 2024-09-19T06:10:35.674 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.602+0000 7fec9b7fe640 1 -- 172.21.15.53:0/2397078035 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feca0032c10 con 0x7feca4106ac0 2024-09-19T06:10:35.674 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.602+0000 7fecac6d4640 1 -- 172.21.15.53:0/2397078035 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feca419b1d0 con 0x7feca4106ac0 2024-09-19T06:10:35.674 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.602+0000 7fec9b7fe640 1 -- 172.21.15.53:0/2397078035 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7feca003d620 con 0x7feca4106ac0 2024-09-19T06:10:35.674 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.602+0000 7fecac6d4640 1 -- 172.21.15.53:0/2397078035 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feca419dec0 con 0x7feca4106ac0 2024-09-19T06:10:35.674 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.603+0000 7fec9b7fe640 1 -- 172.21.15.53:0/2397078035 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 12) ==== 46489+0+0 (secure 0 0 0) 0x7feca003e430 con 0x7feca4106ac0 2024-09-19T06:10:35.674 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.603+0000 7fec9b7fe640 1 --2- 172.21.15.53:0/2397078035 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fec90039290 0x7fec9003b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:35.674 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.603+0000 7fec9b7fe640 1 -- 172.21.15.53:0/2397078035 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7feca006c210 con 0x7feca4106ac0 2024-09-19T06:10:35.674 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.603+0000 7fecac6d4640 1 -- 172.21.15.53:0/2397078035 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feca4106ec0 con 0x7feca4106ac0 2024-09-19T06:10:35.675 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.604+0000 7feca9c6f640 1 --2- 172.21.15.53:0/2397078035 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fec90039290 0x7fec9003b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:35.675 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.604+0000 7feca9c6f640 1 --2- 172.21.15.53:0/2397078035 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fec90039290 0x7fec9003b750 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fec940098c0 tx=0x7fec94006db0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:35.675 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.608+0000 7fec9b7fe640 1 -- 172.21.15.53:0/2397078035 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7feca0045310 con 0x7feca4106ac0 2024-09-19T06:10:35.675 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:34.796+0000 7fecac6d4640 1 -- 172.21.15.53:0/2397078035 --> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7feca419e150 con 0x7fec90039290 2024-09-19T06:10:35.675 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:35.045+0000 7fec9b7fe640 1 -- 172.21.15.53:0/2397078035 <== mgr.14162 v2:172.21.15.53:6800/3684090301 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7feca419e150 con 0x7fec90039290 2024-09-19T06:10:35.675 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:35.047+0000 7fecac6d4640 1 -- 172.21.15.53:0/2397078035 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fec90039290 msgr2=0x7fec9003b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:35.675 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:35.047+0000 7fecac6d4640 1 --2- 172.21.15.53:0/2397078035 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fec90039290 0x7fec9003b750 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fec940098c0 tx=0x7fec94006db0 comp rx=0 tx=0).stop 2024-09-19T06:10:35.675 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:35.047+0000 7fecac6d4640 1 -- 172.21.15.53:0/2397078035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feca4106ac0 msgr2=0x7feca419aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:35.675 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:35.047+0000 7fecac6d4640 1 --2- 172.21.15.53:0/2397078035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feca4106ac0 0x7feca419aa00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feca0006f10 tx=0x7feca0032670 comp rx=0 tx=0).stop 2024-09-19T06:10:35.675 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:35.047+0000 7fecac6d4640 1 -- 172.21.15.53:0/2397078035 shutdown_connections 2024-09-19T06:10:35.675 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:35.047+0000 7fecac6d4640 1 --2- 172.21.15.53:0/2397078035 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fec90039290 0x7fec9003b750 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:35.675 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:35.047+0000 7fecac6d4640 1 --2- 172.21.15.53:0/2397078035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feca4106ac0 0x7feca419aa00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:35.675 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:35.047+0000 7fecac6d4640 1 -- 172.21.15.53:0/2397078035 >> 172.21.15.53:0/2397078035 conn(0x7feca4102330 msgr2=0x7feca4102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:35.676 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:35.047+0000 7fecac6d4640 1 -- 172.21.15.53:0/2397078035 shutdown_connections 2024-09-19T06:10:35.676 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:35.047+0000 7fecac6d4640 1 -- 172.21.15.53:0/2397078035 wait complete. 2024-09-19T06:10:35.676 INFO:teuthology.orchestra.run.smithi053.stdout:Fetching dashboard port number... 2024-09-19T06:10:36.271 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:36 smithi053 ceph-mon[27968]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:10:36.272 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:36 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:36.767 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 8443 2024-09-19T06:10:36.772 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.081+0000 7f9d1b595640 1 Processor -- start 2024-09-19T06:10:36.773 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.082+0000 7f9d1b595640 1 -- start start 2024-09-19T06:10:36.773 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.082+0000 7f9d1b595640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d1407c630 0x7f9d1407aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:36.773 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.082+0000 7f9d1b595640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1407b0e0 con 0x7f9d1407c630 2024-09-19T06:10:36.773 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.082+0000 7f9d19331640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d1407c630 0x7f9d1407aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:36.773 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.082+0000 7f9d19331640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d1407c630 0x7f9d1407aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45146/0 (socket says 172.21.15.53:45146) 2024-09-19T06:10:36.773 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.082+0000 7f9d19331640 1 -- 172.21.15.53:0/3540840389 learned_addr learned my addr 172.21.15.53:0/3540840389 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:36.773 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.082+0000 7f9d19331640 1 -- 172.21.15.53:0/3540840389 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1407b260 con 0x7f9d1407c630 2024-09-19T06:10:36.773 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.083+0000 7f9d19331640 1 --2- 172.21.15.53:0/3540840389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d1407c630 0x7f9d1407aba0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009820 tx=0x7f9d1002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69492a04dfa868b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:36.773 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.083+0000 7f9d0bfff640 1 -- 172.21.15.53:0/3540840389 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f9d10037070 con 0x7f9d1407c630 2024-09-19T06:10:36.773 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.083+0000 7f9d0bfff640 1 -- 172.21.15.53:0/3540840389 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d1002ad40 con 0x7f9d1407c630 2024-09-19T06:10:36.773 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.083+0000 7f9d0bfff640 1 -- 172.21.15.53:0/3540840389 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f9d10030620 con 0x7f9d1407c630 2024-09-19T06:10:36.774 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.084+0000 7f9d1b595640 1 -- 172.21.15.53:0/3540840389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d1407c630 msgr2=0x7f9d1407aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:36.774 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.084+0000 7f9d1b595640 1 --2- 172.21.15.53:0/3540840389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d1407c630 0x7f9d1407aba0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009820 tx=0x7f9d1002a270 comp rx=0 tx=0).stop 2024-09-19T06:10:36.774 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.084+0000 7f9d1b595640 1 -- 172.21.15.53:0/3540840389 shutdown_connections 2024-09-19T06:10:36.774 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.084+0000 7f9d1b595640 1 --2- 172.21.15.53:0/3540840389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d1407c630 0x7f9d1407aba0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.774 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.084+0000 7f9d1b595640 1 -- 172.21.15.53:0/3540840389 >> 172.21.15.53:0/3540840389 conn(0x7f9d14102330 msgr2=0x7f9d14104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:36.774 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.084+0000 7f9d1b595640 1 -- 172.21.15.53:0/3540840389 shutdown_connections 2024-09-19T06:10:36.774 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.084+0000 7f9d1b595640 1 -- 172.21.15.53:0/3540840389 wait complete. 2024-09-19T06:10:36.774 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.084+0000 7f9d1b595640 1 Processor -- start 2024-09-19T06:10:36.774 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.085+0000 7f9d1b595640 1 -- start start 2024-09-19T06:10:36.774 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.085+0000 7f9d1b595640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d1407c630 0x7f9d1419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:36.774 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.085+0000 7f9d1b595640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1410cfc0 con 0x7f9d1407c630 2024-09-19T06:10:36.774 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.085+0000 7f9d19331640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d1407c630 0x7f9d1419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:36.774 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.085+0000 7f9d19331640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d1407c630 0x7f9d1419cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45162/0 (socket says 172.21.15.53:45162) 2024-09-19T06:10:36.774 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.085+0000 7f9d19331640 1 -- 172.21.15.53:0/4041234874 learned_addr learned my addr 172.21.15.53:0/4041234874 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:36.775 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.085+0000 7f9d19331640 1 -- 172.21.15.53:0/4041234874 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1419d120 con 0x7f9d1407c630 2024-09-19T06:10:36.775 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.085+0000 7f9d19331640 1 --2- 172.21.15.53:0/4041234874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d1407c630 0x7f9d1419cbe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9d10000c00 tx=0x7f9d10030fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:36.775 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.085+0000 7f9d0a7fc640 1 -- 172.21.15.53:0/4041234874 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f9d10032a00 con 0x7f9d1407c630 2024-09-19T06:10:36.775 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.085+0000 7f9d0a7fc640 1 -- 172.21.15.53:0/4041234874 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d10032ba0 con 0x7f9d1407c630 2024-09-19T06:10:36.775 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.085+0000 7f9d1b595640 1 -- 172.21.15.53:0/4041234874 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d1419d3b0 con 0x7f9d1407c630 2024-09-19T06:10:36.775 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.086+0000 7f9d0a7fc640 1 -- 172.21.15.53:0/4041234874 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f9d1003d5d0 con 0x7f9d1407c630 2024-09-19T06:10:36.775 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.086+0000 7f9d1b595640 1 -- 172.21.15.53:0/4041234874 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d141a00a0 con 0x7f9d1407c630 2024-09-19T06:10:36.775 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.086+0000 7f9d0a7fc640 1 -- 172.21.15.53:0/4041234874 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 12) ==== 46489+0+0 (secure 0 0 0) 0x7f9d1003e430 con 0x7f9d1407c630 2024-09-19T06:10:36.775 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.086+0000 7f9d1b595640 1 -- 172.21.15.53:0/4041234874 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d1407ac20 con 0x7f9d1407c630 2024-09-19T06:10:36.775 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.087+0000 7f9d0a7fc640 1 --2- 172.21.15.53:0/4041234874 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f9d00039290 0x7f9d0003b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:36.775 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.087+0000 7f9d0a7fc640 1 -- 172.21.15.53:0/4041234874 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9d1006c280 con 0x7f9d1407c630 2024-09-19T06:10:36.775 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.087+0000 7f9d18b30640 1 --2- 172.21.15.53:0/4041234874 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f9d00039290 0x7f9d0003b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:36.775 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.087+0000 7f9d18b30640 1 --2- 172.21.15.53:0/4041234874 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f9d00039290 0x7f9d0003b750 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9d040098c0 tx=0x7f9d04006db0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:36.775 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.091+0000 7f9d0a7fc640 1 -- 172.21.15.53:0/4041234874 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9d100391a0 con 0x7f9d1407c630 2024-09-19T06:10:36.776 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.269+0000 7f9d1b595640 1 -- 172.21.15.53:0/4041234874 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f9d141a0270 con 0x7f9d1407c630 2024-09-19T06:10:36.776 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.270+0000 7f9d0a7fc640 1 -- 172.21.15.53:0/4041234874 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f9d141a0270 con 0x7f9d1407c630 2024-09-19T06:10:36.776 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.272+0000 7f9d1b595640 1 -- 172.21.15.53:0/4041234874 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f9d00039290 msgr2=0x7f9d0003b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:36.776 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.272+0000 7f9d1b595640 1 --2- 172.21.15.53:0/4041234874 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f9d00039290 0x7f9d0003b750 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9d040098c0 tx=0x7f9d04006db0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.776 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.272+0000 7f9d1b595640 1 -- 172.21.15.53:0/4041234874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d1407c630 msgr2=0x7f9d1419cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:36.776 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.272+0000 7f9d1b595640 1 --2- 172.21.15.53:0/4041234874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d1407c630 0x7f9d1419cbe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9d10000c00 tx=0x7f9d10030fc0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.776 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.272+0000 7f9d1b595640 1 -- 172.21.15.53:0/4041234874 shutdown_connections 2024-09-19T06:10:36.776 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.272+0000 7f9d1b595640 1 --2- 172.21.15.53:0/4041234874 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f9d00039290 0x7f9d0003b750 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.776 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.272+0000 7f9d1b595640 1 --2- 172.21.15.53:0/4041234874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d1407c630 0x7f9d1419cbe0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.776 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.272+0000 7f9d1b595640 1 -- 172.21.15.53:0/4041234874 >> 172.21.15.53:0/4041234874 conn(0x7f9d14102330 msgr2=0x7f9d14102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:36.776 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.272+0000 7f9d1b595640 1 -- 172.21.15.53:0/4041234874 shutdown_connections 2024-09-19T06:10:36.776 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:36.272+0000 7f9d1b595640 1 -- 172.21.15.53:0/4041234874 wait complete. 2024-09-19T06:10:36.780 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T06:10:36.780 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout disabled 2024-09-19T06:10:36.787 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T06:10:36.787 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout inactive 2024-09-19T06:10:36.788 INFO:teuthology.orchestra.run.smithi053.stdout:firewalld.service is not enabled 2024-09-19T06:10:36.788 INFO:teuthology.orchestra.run.smithi053.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-19T06:10:36.789 INFO:teuthology.orchestra.run.smithi053.stdout:Ceph Dashboard is now available at: 2024-09-19T06:10:36.789 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:10:36.789 INFO:teuthology.orchestra.run.smithi053.stdout: URL: https://smithi053.front.sepia.ceph.com:8443/ 2024-09-19T06:10:36.790 INFO:teuthology.orchestra.run.smithi053.stdout: User: admin 2024-09-19T06:10:36.790 INFO:teuthology.orchestra.run.smithi053.stdout: Password: rgin5rvlqd 2024-09-19T06:10:36.790 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:10:36.790 INFO:teuthology.orchestra.run.smithi053.stdout:Saving cluster configuration to /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/config directory 2024-09-19T06:10:37.176 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:37 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/4041234874' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-19T06:10:37.995 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.188+0000 7face4548640 1 Processor -- start 2024-09-19T06:10:37.996 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.188+0000 7face4548640 1 -- start start 2024-09-19T06:10:37.996 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.189+0000 7face4548640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facdc107540 0x7facdc109930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:37.996 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.189+0000 7face4548640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7facdc07a730 con 0x7facdc107540 2024-09-19T06:10:37.996 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.189+0000 7face22e4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facdc107540 0x7facdc109930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:37.996 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.189+0000 7face22e4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facdc107540 0x7facdc109930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45166/0 (socket says 172.21.15.53:45166) 2024-09-19T06:10:37.996 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.189+0000 7face22e4640 1 -- 172.21.15.53:0/1982046007 learned_addr learned my addr 172.21.15.53:0/1982046007 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:37.996 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.189+0000 7face22e4640 1 -- 172.21.15.53:0/1982046007 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facdc109e70 con 0x7facdc107540 2024-09-19T06:10:37.996 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.190+0000 7face22e4640 1 --2- 172.21.15.53:0/1982046007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facdc107540 0x7facdc109930 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7facd8009820 tx=0x7facd802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8fd4db197fe1b58 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:37.996 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.190+0000 7face12e2640 1 -- 172.21.15.53:0/1982046007 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7facd8037070 con 0x7facdc107540 2024-09-19T06:10:37.996 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.190+0000 7face12e2640 1 -- 172.21.15.53:0/1982046007 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7facd802ad40 con 0x7facdc107540 2024-09-19T06:10:37.996 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.190+0000 7face12e2640 1 -- 172.21.15.53:0/1982046007 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7facd8030620 con 0x7facdc107540 2024-09-19T06:10:37.996 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.191+0000 7face4548640 1 -- 172.21.15.53:0/1982046007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facdc107540 msgr2=0x7facdc109930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:37.996 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.191+0000 7face4548640 1 --2- 172.21.15.53:0/1982046007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facdc107540 0x7facdc109930 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7facd8009820 tx=0x7facd802a270 comp rx=0 tx=0).stop 2024-09-19T06:10:37.997 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.191+0000 7face4548640 1 -- 172.21.15.53:0/1982046007 shutdown_connections 2024-09-19T06:10:37.997 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.191+0000 7face4548640 1 --2- 172.21.15.53:0/1982046007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facdc107540 0x7facdc109930 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:37.997 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.191+0000 7face4548640 1 -- 172.21.15.53:0/1982046007 >> 172.21.15.53:0/1982046007 conn(0x7facdc1011a0 msgr2=0x7facdc1035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:37.997 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.191+0000 7face4548640 1 -- 172.21.15.53:0/1982046007 shutdown_connections 2024-09-19T06:10:37.997 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.191+0000 7face4548640 1 -- 172.21.15.53:0/1982046007 wait complete. 2024-09-19T06:10:37.997 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.191+0000 7face4548640 1 Processor -- start 2024-09-19T06:10:37.997 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.191+0000 7face4548640 1 -- start start 2024-09-19T06:10:37.997 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.192+0000 7face4548640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facdc107540 0x7facdc19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:37.997 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.192+0000 7face4548640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7facdc10aef0 con 0x7facdc107540 2024-09-19T06:10:37.997 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.192+0000 7face22e4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facdc107540 0x7facdc19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:37.997 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.192+0000 7face22e4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facdc107540 0x7facdc19aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45178/0 (socket says 172.21.15.53:45178) 2024-09-19T06:10:37.997 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.192+0000 7face22e4640 1 -- 172.21.15.53:0/3082448269 learned_addr learned my addr 172.21.15.53:0/3082448269 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:37.997 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.192+0000 7face22e4640 1 -- 172.21.15.53:0/3082448269 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facdc19af90 con 0x7facdc107540 2024-09-19T06:10:37.998 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.192+0000 7face22e4640 1 --2- 172.21.15.53:0/3082448269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facdc107540 0x7facdc19aa50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7facd8000c00 tx=0x7facd8030fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:37.998 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.192+0000 7faccf7fe640 1 -- 172.21.15.53:0/3082448269 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7facd8032a00 con 0x7facdc107540 2024-09-19T06:10:37.998 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.192+0000 7faccf7fe640 1 -- 172.21.15.53:0/3082448269 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7facd8032ba0 con 0x7facdc107540 2024-09-19T06:10:37.998 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.192+0000 7faccf7fe640 1 -- 172.21.15.53:0/3082448269 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7facd803d5d0 con 0x7facdc107540 2024-09-19T06:10:37.998 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.192+0000 7face4548640 1 -- 172.21.15.53:0/3082448269 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facdc19b220 con 0x7facdc107540 2024-09-19T06:10:37.998 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.192+0000 7face4548640 1 -- 172.21.15.53:0/3082448269 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facdc19df10 con 0x7facdc107540 2024-09-19T06:10:37.998 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.193+0000 7faccf7fe640 1 -- 172.21.15.53:0/3082448269 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46532+0+0 (secure 0 0 0) 0x7facd803e430 con 0x7facdc107540 2024-09-19T06:10:37.998 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.193+0000 7face4548640 1 -- 172.21.15.53:0/3082448269 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facdc1079a0 con 0x7facdc107540 2024-09-19T06:10:37.998 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.193+0000 7faccf7fe640 1 --2- 172.21.15.53:0/3082448269 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7facc8039330 0x7facc803b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:37.998 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.194+0000 7faccf7fe640 1 -- 172.21.15.53:0/3082448269 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7facd806c340 con 0x7facdc107540 2024-09-19T06:10:37.998 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.194+0000 7face1ae3640 1 --2- 172.21.15.53:0/3082448269 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7facc8039330 0x7facc803b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:37.998 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.194+0000 7face1ae3640 1 --2- 172.21.15.53:0/3082448269 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7facc8039330 0x7facc803b7f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7facd00098c0 tx=0x7facd0006db0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:37.998 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.198+0000 7faccf7fe640 1 -- 172.21.15.53:0/3082448269 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7facd803ce50 con 0x7facdc107540 2024-09-19T06:10:37.998 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.466+0000 7face4548640 1 -- 172.21.15.53:0/3082448269 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7facdc19e5b0 con 0x7facdc107540 2024-09-19T06:10:37.999 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.469+0000 7faccf7fe640 1 -- 172.21.15.53:0/3082448269 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v34)=0 set mgr/dashboard/cluster/status v34) ==== 153+0+0 (secure 0 0 0) 0x7facd803e720 con 0x7facdc107540 2024-09-19T06:10:37.999 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-19T06:10:37.999 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.472+0000 7face4548640 1 -- 172.21.15.53:0/3082448269 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7facc8039330 msgr2=0x7facc803b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:37.999 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.472+0000 7face4548640 1 --2- 172.21.15.53:0/3082448269 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7facc8039330 0x7facc803b7f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7facd00098c0 tx=0x7facd0006db0 comp rx=0 tx=0).stop 2024-09-19T06:10:37.999 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.472+0000 7face4548640 1 -- 172.21.15.53:0/3082448269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facdc107540 msgr2=0x7facdc19aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:37.999 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.472+0000 7face4548640 1 --2- 172.21.15.53:0/3082448269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facdc107540 0x7facdc19aa50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7facd8000c00 tx=0x7facd8030fc0 comp rx=0 tx=0).stop 2024-09-19T06:10:37.999 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.472+0000 7face4548640 1 -- 172.21.15.53:0/3082448269 shutdown_connections 2024-09-19T06:10:37.999 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.472+0000 7face4548640 1 --2- 172.21.15.53:0/3082448269 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7facc8039330 0x7facc803b7f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:37.999 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.472+0000 7face4548640 1 --2- 172.21.15.53:0/3082448269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facdc107540 0x7facdc19aa50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:37.999 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.472+0000 7face4548640 1 -- 172.21.15.53:0/3082448269 >> 172.21.15.53:0/3082448269 conn(0x7facdc1011a0 msgr2=0x7facdc101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:37.999 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.472+0000 7face4548640 1 -- 172.21.15.53:0/3082448269 shutdown_connections 2024-09-19T06:10:37.999 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-09-19T06:10:37.472+0000 7face4548640 1 -- 172.21.15.53:0/3082448269 wait complete. 2024-09-19T06:10:37.999 INFO:teuthology.orchestra.run.smithi053.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-19T06:10:37.999 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:10:38.000 INFO:teuthology.orchestra.run.smithi053.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-19T06:10:38.000 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:10:38.000 INFO:teuthology.orchestra.run.smithi053.stdout:Or, if you are only running a single cluster on this host: 2024-09-19T06:10:38.000 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:10:38.000 INFO:teuthology.orchestra.run.smithi053.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-19T06:10:38.000 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:10:38.000 INFO:teuthology.orchestra.run.smithi053.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-19T06:10:38.000 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:10:38.000 INFO:teuthology.orchestra.run.smithi053.stdout: ceph telemetry on 2024-09-19T06:10:38.000 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:10:38.000 INFO:teuthology.orchestra.run.smithi053.stdout:For more information see: 2024-09-19T06:10:38.000 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:10:38.000 INFO:teuthology.orchestra.run.smithi053.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-19T06:10:38.000 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:10:38.001 INFO:teuthology.orchestra.run.smithi053.stdout:Bootstrap complete. 2024-09-19T06:10:38.035 INFO:tasks.cephadm:Fetching config... 2024-09-19T06:10:38.035 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-19T06:10:38.036 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-19T06:10:38.053 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-19T06:10:38.053 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-19T06:10:38.053 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-19T06:10:38.108 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-19T06:10:38.109 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-19T06:10:38.109 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/keyring of=/dev/stdout 2024-09-19T06:10:38.175 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-19T06:10:38.175 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-19T06:10:38.175 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-19T06:10:38.231 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-19T06:10:38.231 DEBUG:teuthology.orchestra.run.smithi053:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBb9VgqIg+Ix/qTJ2Gp8aKvxKYTwfZu1J68PwIDlf1t856NHsgy5gxsF+JDUdg6NG4oLn+1aH/HMb/RGHTToTwoliFX7nKpclWmXUByg3eBlQc4vk9Pj54eHOTe3bt27E50vWmxYuWCgpb/AW43dHg1n12MdzaDN0xvXNq4EP7pSxQX+Wb6nXhXHbsBjRTi/ti/O4JGdDf4azQ8xjjTkREOchIFh97RgsJEiR/6bvXWpoATQS4YRn4WyNMPCqC11ibwagnHmVDiZ9yNaZg1lcGmon+K0BuLbtq59SrgQ/uHsO9JzhyRpPV8Hlsdj2/l/IrOMIC5HXNvV5K9209JVQ6JCK59t0otmefAcyxNaH8T6ifCzj7O/GitUZDi6JmBuPsTuGd5kdMKrOYWHGKvCYEvB1OR8L5rDYVZtmrDvGqLk3Nr5uu+4LRZekzflPdr9MoVX90qMaaqZLZxL0u91P1bGOOmrKDdX58TB1zNcQjxjUOBfKI5ZqhfS+xEVNoB5E= ceph-9f2876d0-764d-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T06:10:38.293 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:38 smithi053 ceph-mon[27968]: mgrmap e13: smithi053.trakvg(active, since 6s) 2024-09-19T06:10:38.293 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:38 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/3082448269' entity='client.admin' 2024-09-19T06:10:38.322 INFO:teuthology.orchestra.run.smithi053.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBb9VgqIg+Ix/qTJ2Gp8aKvxKYTwfZu1J68PwIDlf1t856NHsgy5gxsF+JDUdg6NG4oLn+1aH/HMb/RGHTToTwoliFX7nKpclWmXUByg3eBlQc4vk9Pj54eHOTe3bt27E50vWmxYuWCgpb/AW43dHg1n12MdzaDN0xvXNq4EP7pSxQX+Wb6nXhXHbsBjRTi/ti/O4JGdDf4azQ8xjjTkREOchIFh97RgsJEiR/6bvXWpoATQS4YRn4WyNMPCqC11ibwagnHmVDiZ9yNaZg1lcGmon+K0BuLbtq59SrgQ/uHsO9JzhyRpPV8Hlsdj2/l/IrOMIC5HXNvV5K9209JVQ6JCK59t0otmefAcyxNaH8T6ifCzj7O/GitUZDi6JmBuPsTuGd5kdMKrOYWHGKvCYEvB1OR8L5rDYVZtmrDvGqLk3Nr5uu+4LRZekzflPdr9MoVX90qMaaqZLZxL0u91P1bGOOmrKDdX58TB1zNcQjxjUOBfKI5ZqhfS+xEVNoB5E= ceph-9f2876d0-764d-11ef-bceb-c7b262605968 2024-09-19T06:10:38.334 DEBUG:teuthology.orchestra.run.smithi155:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBb9VgqIg+Ix/qTJ2Gp8aKvxKYTwfZu1J68PwIDlf1t856NHsgy5gxsF+JDUdg6NG4oLn+1aH/HMb/RGHTToTwoliFX7nKpclWmXUByg3eBlQc4vk9Pj54eHOTe3bt27E50vWmxYuWCgpb/AW43dHg1n12MdzaDN0xvXNq4EP7pSxQX+Wb6nXhXHbsBjRTi/ti/O4JGdDf4azQ8xjjTkREOchIFh97RgsJEiR/6bvXWpoATQS4YRn4WyNMPCqC11ibwagnHmVDiZ9yNaZg1lcGmon+K0BuLbtq59SrgQ/uHsO9JzhyRpPV8Hlsdj2/l/IrOMIC5HXNvV5K9209JVQ6JCK59t0otmefAcyxNaH8T6ifCzj7O/GitUZDi6JmBuPsTuGd5kdMKrOYWHGKvCYEvB1OR8L5rDYVZtmrDvGqLk3Nr5uu+4LRZekzflPdr9MoVX90qMaaqZLZxL0u91P1bGOOmrKDdX58TB1zNcQjxjUOBfKI5ZqhfS+xEVNoB5E= ceph-9f2876d0-764d-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T06:10:38.387 INFO:teuthology.orchestra.run.smithi155.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBb9VgqIg+Ix/qTJ2Gp8aKvxKYTwfZu1J68PwIDlf1t856NHsgy5gxsF+JDUdg6NG4oLn+1aH/HMb/RGHTToTwoliFX7nKpclWmXUByg3eBlQc4vk9Pj54eHOTe3bt27E50vWmxYuWCgpb/AW43dHg1n12MdzaDN0xvXNq4EP7pSxQX+Wb6nXhXHbsBjRTi/ti/O4JGdDf4azQ8xjjTkREOchIFh97RgsJEiR/6bvXWpoATQS4YRn4WyNMPCqC11ibwagnHmVDiZ9yNaZg1lcGmon+K0BuLbtq59SrgQ/uHsO9JzhyRpPV8Hlsdj2/l/IrOMIC5HXNvV5K9209JVQ6JCK59t0otmefAcyxNaH8T6ifCzj7O/GitUZDi6JmBuPsTuGd5kdMKrOYWHGKvCYEvB1OR8L5rDYVZtmrDvGqLk3Nr5uu+4LRZekzflPdr9MoVX90qMaaqZLZxL0u91P1bGOOmrKDdX58TB1zNcQjxjUOBfKI5ZqhfS+xEVNoB5E= ceph-9f2876d0-764d-11ef-bceb-c7b262605968 2024-09-19T06:10:38.400 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-19T06:10:38.653 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:10:39.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.157+0000 7f0bfa6df640 1 -- 172.21.15.53:0/613129472 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bf4101600 msgr2=0x7f0bf4101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:39.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.157+0000 7f0bfa6df640 1 --2- 172.21.15.53:0/613129472 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bf4101600 0x7f0bf4101a00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0be4009880 tx=0x7f0be402a5d0 comp rx=0 tx=0).stop 2024-09-19T06:10:39.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.157+0000 7f0bfa6df640 1 -- 172.21.15.53:0/613129472 shutdown_connections 2024-09-19T06:10:39.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.157+0000 7f0bfa6df640 1 --2- 172.21.15.53:0/613129472 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bf4101600 0x7f0bf4101a00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:39.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.157+0000 7f0bfa6df640 1 -- 172.21.15.53:0/613129472 >> 172.21.15.53:0/613129472 conn(0x7f0bf40fcdb0 msgr2=0x7f0bf40ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:39.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.157+0000 7f0bfa6df640 1 -- 172.21.15.53:0/613129472 shutdown_connections 2024-09-19T06:10:39.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.157+0000 7f0bfa6df640 1 -- 172.21.15.53:0/613129472 wait complete. 2024-09-19T06:10:39.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.158+0000 7f0bfa6df640 1 Processor -- start 2024-09-19T06:10:39.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.158+0000 7f0bfa6df640 1 -- start start 2024-09-19T06:10:39.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.159+0000 7f0bfa6df640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bf4101600 0x7f0bf4196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:39.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.159+0000 7f0bfa6df640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf41075c0 con 0x7f0bf4101600 2024-09-19T06:10:39.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.159+0000 7f0bf3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bf4101600 0x7f0bf4196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:39.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.159+0000 7f0bf3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bf4101600 0x7f0bf4196cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45200/0 (socket says 172.21.15.53:45200) 2024-09-19T06:10:39.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.159+0000 7f0bf3fff640 1 -- 172.21.15.53:0/3086755862 learned_addr learned my addr 172.21.15.53:0/3086755862 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:39.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.159+0000 7f0bf3fff640 1 -- 172.21.15.53:0/3086755862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf4197200 con 0x7f0bf4101600 2024-09-19T06:10:39.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.159+0000 7f0bf3fff640 1 --2- 172.21.15.53:0/3086755862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bf4101600 0x7f0bf4196cc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0be40046e0 tx=0x7f0be4004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:39.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.159+0000 7f0bf17fa640 1 -- 172.21.15.53:0/3086755862 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f0be403f070 con 0x7f0bf4101600 2024-09-19T06:10:39.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.160+0000 7f0bf17fa640 1 -- 172.21.15.53:0/3086755862 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0be4003aa0 con 0x7f0bf4101600 2024-09-19T06:10:39.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.160+0000 7f0bf17fa640 1 -- 172.21.15.53:0/3086755862 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f0be4035030 con 0x7f0bf4101600 2024-09-19T06:10:39.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.160+0000 7f0bfa6df640 1 -- 172.21.15.53:0/3086755862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bf4197490 con 0x7f0bf4101600 2024-09-19T06:10:39.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.160+0000 7f0bfa6df640 1 -- 172.21.15.53:0/3086755862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bf41978b0 con 0x7f0bf4101600 2024-09-19T06:10:39.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.161+0000 7f0bf17fa640 1 -- 172.21.15.53:0/3086755862 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46532+0+0 (secure 0 0 0) 0x7f0be4042070 con 0x7f0bf4101600 2024-09-19T06:10:39.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.161+0000 7f0bf17fa640 1 --2- 172.21.15.53:0/3086755862 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f0bdc039330 0x7f0bdc03b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:39.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.161+0000 7f0bfa6df640 1 -- 172.21.15.53:0/3086755862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bf4107d40 con 0x7f0bf4101600 2024-09-19T06:10:39.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.161+0000 7f0bf17fa640 1 -- 172.21.15.53:0/3086755862 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0be406d1c0 con 0x7f0bf4101600 2024-09-19T06:10:39.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.161+0000 7f0bf37fe640 1 --2- 172.21.15.53:0/3086755862 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f0bdc039330 0x7f0bdc03b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:39.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.162+0000 7f0bf37fe640 1 --2- 172.21.15.53:0/3086755862 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f0bdc039330 0x7f0bdc03b7f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0be80098c0 tx=0x7f0be8006db0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:39.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.166+0000 7f0bf17fa640 1 -- 172.21.15.53:0/3086755862 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0be4042360 con 0x7f0bf4101600 2024-09-19T06:10:39.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.345+0000 7f0bfa6df640 1 -- 172.21.15.53:0/3086755862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f0bf402d9b0 con 0x7f0bf4101600 2024-09-19T06:10:39.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.348+0000 7f0bf17fa640 1 -- 172.21.15.53:0/3086755862 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f0be402f300 con 0x7f0bf4101600 2024-09-19T06:10:39.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.350+0000 7f0bfa6df640 1 -- 172.21.15.53:0/3086755862 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f0bdc039330 msgr2=0x7f0bdc03b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:39.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.350+0000 7f0bfa6df640 1 --2- 172.21.15.53:0/3086755862 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f0bdc039330 0x7f0bdc03b7f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0be80098c0 tx=0x7f0be8006db0 comp rx=0 tx=0).stop 2024-09-19T06:10:39.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.350+0000 7f0bfa6df640 1 -- 172.21.15.53:0/3086755862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bf4101600 msgr2=0x7f0bf4196cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:39.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.350+0000 7f0bfa6df640 1 --2- 172.21.15.53:0/3086755862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bf4101600 0x7f0bf4196cc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0be40046e0 tx=0x7f0be4004710 comp rx=0 tx=0).stop 2024-09-19T06:10:39.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.350+0000 7f0bfa6df640 1 -- 172.21.15.53:0/3086755862 shutdown_connections 2024-09-19T06:10:39.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.350+0000 7f0bfa6df640 1 --2- 172.21.15.53:0/3086755862 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f0bdc039330 0x7f0bdc03b7f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:39.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.350+0000 7f0bfa6df640 1 --2- 172.21.15.53:0/3086755862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bf4101600 0x7f0bf4196cc0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:39.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.350+0000 7f0bfa6df640 1 -- 172.21.15.53:0/3086755862 >> 172.21.15.53:0/3086755862 conn(0x7f0bf40fcdb0 msgr2=0x7f0bf40fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:39.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.350+0000 7f0bfa6df640 1 -- 172.21.15.53:0/3086755862 shutdown_connections 2024-09-19T06:10:39.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:39.350+0000 7f0bfa6df640 1 -- 172.21.15.53:0/3086755862 wait complete. 2024-09-19T06:10:40.431 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:40 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/3086755862' entity='client.admin' 2024-09-19T06:10:40.431 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:40 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:40.432 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:40 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:40.432 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:40 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd/host:smithi053", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:10:40.432 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:40 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:40.432 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:40 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi053", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-19T06:10:40.432 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:40 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi053", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-19T06:10:40.432 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:40 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:10:40.432 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:40 smithi053 ceph-mon[27968]: Deploying daemon ceph-exporter.smithi053 on smithi053 2024-09-19T06:10:40.895 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-19T06:10:40.895 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-19T06:10:41.709 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:10:43.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.077+0000 7fece2aa3640 1 -- 172.21.15.53:0/4012497694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecdc1015e0 msgr2=0x7fecdc1019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:43.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.077+0000 7fece2aa3640 1 --2- 172.21.15.53:0/4012497694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecdc1015e0 0x7fecdc1019e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fecd4009880 tx=0x7fecd402a5d0 comp rx=0 tx=0).stop 2024-09-19T06:10:43.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.077+0000 7fece2aa3640 1 -- 172.21.15.53:0/4012497694 shutdown_connections 2024-09-19T06:10:43.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.077+0000 7fece2aa3640 1 --2- 172.21.15.53:0/4012497694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecdc1015e0 0x7fecdc1019e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:43.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.077+0000 7fece2aa3640 1 -- 172.21.15.53:0/4012497694 >> 172.21.15.53:0/4012497694 conn(0x7fecdc0fcd70 msgr2=0x7fecdc0ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:43.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.077+0000 7fece2aa3640 1 -- 172.21.15.53:0/4012497694 shutdown_connections 2024-09-19T06:10:43.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.078+0000 7fece2aa3640 1 -- 172.21.15.53:0/4012497694 wait complete. 2024-09-19T06:10:43.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.078+0000 7fece2aa3640 1 Processor -- start 2024-09-19T06:10:43.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.078+0000 7fece2aa3640 1 -- start start 2024-09-19T06:10:43.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.078+0000 7fece2aa3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecdc1015e0 0x7fecdc075eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:43.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.078+0000 7fece2aa3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fecdc1075a0 con 0x7fecdc1015e0 2024-09-19T06:10:43.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.079+0000 7fece083f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecdc1015e0 0x7fecdc075eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:43.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.079+0000 7fece083f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecdc1015e0 0x7fecdc075eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43708/0 (socket says 172.21.15.53:43708) 2024-09-19T06:10:43.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.079+0000 7fece083f640 1 -- 172.21.15.53:0/3848317957 learned_addr learned my addr 172.21.15.53:0/3848317957 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:43.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.079+0000 7fece083f640 1 -- 172.21.15.53:0/3848317957 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecdc0763f0 con 0x7fecdc1015e0 2024-09-19T06:10:43.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.079+0000 7fece083f640 1 --2- 172.21.15.53:0/3848317957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecdc1015e0 0x7fecdc075eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fecd4005ab0 tx=0x7fecd40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:43.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.079+0000 7fecd9ffb640 1 -- 172.21.15.53:0/3848317957 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fecd403f070 con 0x7fecdc1015e0 2024-09-19T06:10:43.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.079+0000 7fecd9ffb640 1 -- 172.21.15.53:0/3848317957 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecd4003aa0 con 0x7fecdc1015e0 2024-09-19T06:10:43.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.079+0000 7fece2aa3640 1 -- 172.21.15.53:0/3848317957 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecdc0745e0 con 0x7fecdc1015e0 2024-09-19T06:10:43.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.079+0000 7fecd9ffb640 1 -- 172.21.15.53:0/3848317957 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fecd4035030 con 0x7fecdc1015e0 2024-09-19T06:10:43.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.079+0000 7fece2aa3640 1 -- 172.21.15.53:0/3848317957 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecdc074b20 con 0x7fecdc1015e0 2024-09-19T06:10:43.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.080+0000 7fecd9ffb640 1 -- 172.21.15.53:0/3848317957 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46532+0+0 (secure 0 0 0) 0x7fecd4042070 con 0x7fecdc1015e0 2024-09-19T06:10:43.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.081+0000 7fecd9ffb640 1 --2- 172.21.15.53:0/3848317957 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fecc80392d0 0x7fecc803b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:43.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.081+0000 7fecd9ffb640 1 -- 172.21.15.53:0/3848317957 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fecd406d130 con 0x7fecdc1015e0 2024-09-19T06:10:43.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.081+0000 7fecdbfff640 1 --2- 172.21.15.53:0/3848317957 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fecc80392d0 0x7fecc803b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:43.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.081+0000 7fece2aa3640 1 -- 172.21.15.53:0/3848317957 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecdc107d20 con 0x7fecdc1015e0 2024-09-19T06:10:43.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.081+0000 7fecdbfff640 1 --2- 172.21.15.53:0/3848317957 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fecc80392d0 0x7fecc803b790 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7feccc00ad30 tx=0x7feccc0093f0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:43.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.086+0000 7fecd9ffb640 1 -- 172.21.15.53:0/3848317957 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fecd4042360 con 0x7fecdc1015e0 2024-09-19T06:10:43.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.284+0000 7fece2aa3640 1 -- 172.21.15.53:0/3848317957 --> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fecdc18fe40 con 0x7fecc80392d0 2024-09-19T06:10:43.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.288+0000 7fecd9ffb640 1 -- 172.21.15.53:0/3848317957 <== mgr.14162 v2:172.21.15.53:6800/3684090301 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fecdc18fe40 con 0x7fecc80392d0 2024-09-19T06:10:43.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.290+0000 7fece2aa3640 1 -- 172.21.15.53:0/3848317957 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fecc80392d0 msgr2=0x7fecc803b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:43.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.290+0000 7fece2aa3640 1 --2- 172.21.15.53:0/3848317957 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fecc80392d0 0x7fecc803b790 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7feccc00ad30 tx=0x7feccc0093f0 comp rx=0 tx=0).stop 2024-09-19T06:10:43.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.290+0000 7fece2aa3640 1 -- 172.21.15.53:0/3848317957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecdc1015e0 msgr2=0x7fecdc075eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:43.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.290+0000 7fece2aa3640 1 --2- 172.21.15.53:0/3848317957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecdc1015e0 0x7fecdc075eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fecd4005ab0 tx=0x7fecd40048f0 comp rx=0 tx=0).stop 2024-09-19T06:10:43.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.290+0000 7fece2aa3640 1 -- 172.21.15.53:0/3848317957 shutdown_connections 2024-09-19T06:10:43.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.290+0000 7fece2aa3640 1 --2- 172.21.15.53:0/3848317957 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fecc80392d0 0x7fecc803b790 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:43.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.290+0000 7fece2aa3640 1 --2- 172.21.15.53:0/3848317957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecdc1015e0 0x7fecdc075eb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:43.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.290+0000 7fece2aa3640 1 -- 172.21.15.53:0/3848317957 >> 172.21.15.53:0/3848317957 conn(0x7fecdc0fcd70 msgr2=0x7fecdc0fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:43.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.290+0000 7fece2aa3640 1 -- 172.21.15.53:0/3848317957 shutdown_connections 2024-09-19T06:10:43.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:43.290+0000 7fece2aa3640 1 -- 172.21.15.53:0/3848317957 wait complete. 2024-09-19T06:10:43.869 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi155 2024-09-19T06:10:43.869 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-19T06:10:43.869 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/etc/ceph/ceph.conf 2024-09-19T06:10:43.888 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-19T06:10:43.888 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:10:43.946 INFO:tasks.cephadm:Adding host smithi155 to orchestrator... 2024-09-19T06:10:43.946 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph orch host add smithi155 2024-09-19T06:10:44.444 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:10:44.560 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:44 smithi053 ceph-mon[27968]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:10:44.560 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:44 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:44.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:44.822+0000 7f20ffb09640 1 -- 172.21.15.53:0/1014288556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20f8102070 msgr2=0x7f20f8104460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:44.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:44.822+0000 7f20ffb09640 1 --2- 172.21.15.53:0/1014288556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20f8102070 0x7f20f8104460 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009880 tx=0x7f20f402a5d0 comp rx=0 tx=0).stop 2024-09-19T06:10:44.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:44.822+0000 7f20ffb09640 1 -- 172.21.15.53:0/1014288556 shutdown_connections 2024-09-19T06:10:44.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:44.822+0000 7f20ffb09640 1 --2- 172.21.15.53:0/1014288556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20f8102070 0x7f20f8104460 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:44.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:44.822+0000 7f20ffb09640 1 -- 172.21.15.53:0/1014288556 >> 172.21.15.53:0/1014288556 conn(0x7f20f80fbc90 msgr2=0x7f20f80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:44.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:44.822+0000 7f20ffb09640 1 -- 172.21.15.53:0/1014288556 shutdown_connections 2024-09-19T06:10:44.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:44.822+0000 7f20ffb09640 1 -- 172.21.15.53:0/1014288556 wait complete. 2024-09-19T06:10:44.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:44.823+0000 7f20ffb09640 1 Processor -- start 2024-09-19T06:10:44.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:44.823+0000 7f20ffb09640 1 -- start start 2024-09-19T06:10:44.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:44.823+0000 7f20ffb09640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20f8102070 0x7f20f81018f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:44.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:44.823+0000 7f20ffb09640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f20f8107530 con 0x7f20f8102070 2024-09-19T06:10:44.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:44.823+0000 7f20fd8a5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20f8102070 0x7f20f81018f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:44.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:44.823+0000 7f20fd8a5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20f8102070 0x7f20f81018f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43730/0 (socket says 172.21.15.53:43730) 2024-09-19T06:10:44.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:44.823+0000 7f20fd8a5640 1 -- 172.21.15.53:0/1763159615 learned_addr learned my addr 172.21.15.53:0/1763159615 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:44.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:44.824+0000 7f20fd8a5640 1 -- 172.21.15.53:0/1763159615 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f8101e30 con 0x7f20f8102070 2024-09-19T06:10:44.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:44.824+0000 7f20fd8a5640 1 --2- 172.21.15.53:0/1763159615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20f8102070 0x7f20f81018f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f20f4006010 tx=0x7f20f40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:44.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:44.824+0000 7f20eeffd640 1 -- 172.21.15.53:0/1763159615 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f20f403f070 con 0x7f20f8102070 2024-09-19T06:10:44.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:44.824+0000 7f20ffb09640 1 -- 172.21.15.53:0/1763159615 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f8100020 con 0x7f20f8102070 2024-09-19T06:10:44.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:44.824+0000 7f20eeffd640 1 -- 172.21.15.53:0/1763159615 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20f4003aa0 con 0x7f20f8102070 2024-09-19T06:10:44.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:44.824+0000 7f20eeffd640 1 -- 172.21.15.53:0/1763159615 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f20f4035030 con 0x7f20f8102070 2024-09-19T06:10:44.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:44.824+0000 7f20ffb09640 1 -- 172.21.15.53:0/1763159615 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f8100560 con 0x7f20f8102070 2024-09-19T06:10:44.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:44.825+0000 7f20eeffd640 1 -- 172.21.15.53:0/1763159615 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46532+0+0 (secure 0 0 0) 0x7f20f4042070 con 0x7f20f8102070 2024-09-19T06:10:44.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:44.825+0000 7f20eeffd640 1 --2- 172.21.15.53:0/1763159615 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f20e4039330 0x7f20e403b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:44.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:44.825+0000 7f20eeffd640 1 -- 172.21.15.53:0/1763159615 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f20f406dfd0 con 0x7f20f8102070 2024-09-19T06:10:44.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:44.825+0000 7f20ffb09640 1 -- 172.21.15.53:0/1763159615 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20f8107cb0 con 0x7f20f8102070 2024-09-19T06:10:44.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:44.826+0000 7f20fd0a4640 1 --2- 172.21.15.53:0/1763159615 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f20e4039330 0x7f20e403b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:44.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:44.826+0000 7f20fd0a4640 1 --2- 172.21.15.53:0/1763159615 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f20e4039330 0x7f20e403b7f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f20f000ad30 tx=0x7f20f00093f0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:44.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:44.831+0000 7f20eeffd640 1 -- 172.21.15.53:0/1763159615 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f20f40433b0 con 0x7f20f8102070 2024-09-19T06:10:45.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:45.025+0000 7f20ffb09640 1 -- 172.21.15.53:0/1763159615 --> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi155", "target": ["mon-mgr", ""]}) -- 0x7f20f818ff00 con 0x7f20e4039330 2024-09-19T06:10:45.601 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:45 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:45.601 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:45 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:45.601 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:45 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:45.601 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:45 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:45.601 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:45 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi053", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-19T06:10:45.601 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:45 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi053", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-09-19T06:10:45.602 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:45 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:10:45.602 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:45 smithi053 ceph-mon[27968]: Deploying daemon crash.smithi053 on smithi053 2024-09-19T06:10:45.602 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:45 smithi053 ceph-mon[27968]: from='client.14187 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi155", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:10:45.602 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:45 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:48.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:47 smithi053 ceph-mon[27968]: Deploying cephadm binary to smithi155 2024-09-19T06:10:48.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:47 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:48.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:47 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:48.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:47 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:48.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:47 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:48.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:47 smithi053 ceph-mon[27968]: Deploying daemon node-exporter.smithi053 on smithi053 2024-09-19T06:10:49.031 INFO:teuthology.orchestra.run.smithi053.stdout:Added host 'smithi155' with addr '172.21.15.155' 2024-09-19T06:10:49.031 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:49.027+0000 7f20eeffd640 1 -- 172.21.15.53:0/1763159615 <== mgr.14162 v2:172.21.15.53:6800/3684090301 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f20f818ff00 con 0x7f20e4039330 2024-09-19T06:10:49.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:49.030+0000 7f20ffb09640 1 -- 172.21.15.53:0/1763159615 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f20e4039330 msgr2=0x7f20e403b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:49.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:49.030+0000 7f20ffb09640 1 --2- 172.21.15.53:0/1763159615 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f20e4039330 0x7f20e403b7f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f20f000ad30 tx=0x7f20f00093f0 comp rx=0 tx=0).stop 2024-09-19T06:10:49.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:49.030+0000 7f20ffb09640 1 -- 172.21.15.53:0/1763159615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20f8102070 msgr2=0x7f20f81018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:49.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:49.030+0000 7f20ffb09640 1 --2- 172.21.15.53:0/1763159615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20f8102070 0x7f20f81018f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f20f4006010 tx=0x7f20f40048f0 comp rx=0 tx=0).stop 2024-09-19T06:10:49.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:49.030+0000 7f20ffb09640 1 -- 172.21.15.53:0/1763159615 shutdown_connections 2024-09-19T06:10:49.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:49.030+0000 7f20ffb09640 1 --2- 172.21.15.53:0/1763159615 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f20e4039330 0x7f20e403b7f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:49.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:49.030+0000 7f20ffb09640 1 --2- 172.21.15.53:0/1763159615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20f8102070 0x7f20f81018f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:49.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:49.030+0000 7f20ffb09640 1 -- 172.21.15.53:0/1763159615 >> 172.21.15.53:0/1763159615 conn(0x7f20f80fbc90 msgr2=0x7f20f80fc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:49.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:49.030+0000 7f20ffb09640 1 -- 172.21.15.53:0/1763159615 shutdown_connections 2024-09-19T06:10:49.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:49.030+0000 7f20ffb09640 1 -- 172.21.15.53:0/1763159615 wait complete. 2024-09-19T06:10:50.181 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:50 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:50.182 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:50 smithi053 ceph-mon[27968]: Added host smithi155 2024-09-19T06:10:50.329 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T06:10:50.769 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:10:51.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.235+0000 7fdd96be3640 1 -- 172.21.15.53:0/1533339055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd90100510 msgr2=0x7fdd90100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:51.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.235+0000 7fdd96be3640 1 --2- 172.21.15.53:0/1533339055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd90100510 0x7fdd90100910 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdd84009880 tx=0x7fdd8402a5d0 comp rx=0 tx=0).stop 2024-09-19T06:10:51.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.235+0000 7fdd96be3640 1 -- 172.21.15.53:0/1533339055 shutdown_connections 2024-09-19T06:10:51.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.235+0000 7fdd96be3640 1 --2- 172.21.15.53:0/1533339055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd90100510 0x7fdd90100910 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:51.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.235+0000 7fdd96be3640 1 -- 172.21.15.53:0/1533339055 >> 172.21.15.53:0/1533339055 conn(0x7fdd900fbc80 msgr2=0x7fdd900fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:51.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.235+0000 7fdd96be3640 1 -- 172.21.15.53:0/1533339055 shutdown_connections 2024-09-19T06:10:51.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.235+0000 7fdd96be3640 1 -- 172.21.15.53:0/1533339055 wait complete. 2024-09-19T06:10:51.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.236+0000 7fdd96be3640 1 Processor -- start 2024-09-19T06:10:51.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.236+0000 7fdd96be3640 1 -- start start 2024-09-19T06:10:51.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.236+0000 7fdd96be3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd90100510 0x7fdd9006a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:51.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.236+0000 7fdd96be3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901064d0 con 0x7fdd90100510 2024-09-19T06:10:51.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.236+0000 7fdd9497f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd90100510 0x7fdd9006a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:51.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.236+0000 7fdd9497f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd90100510 0x7fdd9006a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43766/0 (socket says 172.21.15.53:43766) 2024-09-19T06:10:51.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.236+0000 7fdd9497f640 1 -- 172.21.15.53:0/2954602996 learned_addr learned my addr 172.21.15.53:0/2954602996 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:51.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.236+0000 7fdd9497f640 1 -- 172.21.15.53:0/2954602996 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd901999f0 con 0x7fdd90100510 2024-09-19T06:10:51.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.237+0000 7fdd9497f640 1 --2- 172.21.15.53:0/2954602996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd90100510 0x7fdd9006a950 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdd840046e0 tx=0x7fdd84004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:51.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.237+0000 7fdd8dffb640 1 -- 172.21.15.53:0/2954602996 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fdd8403f070 con 0x7fdd90100510 2024-09-19T06:10:51.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.237+0000 7fdd8dffb640 1 -- 172.21.15.53:0/2954602996 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd84003aa0 con 0x7fdd90100510 2024-09-19T06:10:51.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.237+0000 7fdd96be3640 1 -- 172.21.15.53:0/2954602996 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd90199bc0 con 0x7fdd90100510 2024-09-19T06:10:51.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.237+0000 7fdd8dffb640 1 -- 172.21.15.53:0/2954602996 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fdd84035030 con 0x7fdd90100510 2024-09-19T06:10:51.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.237+0000 7fdd96be3640 1 -- 172.21.15.53:0/2954602996 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd90199f60 con 0x7fdd90100510 2024-09-19T06:10:51.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.238+0000 7fdd8dffb640 1 -- 172.21.15.53:0/2954602996 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46532+0+0 (secure 0 0 0) 0x7fdd84042070 con 0x7fdd90100510 2024-09-19T06:10:51.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.238+0000 7fdd8dffb640 1 --2- 172.21.15.53:0/2954602996 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fdd80039380 0x7fdd8003b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:51.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.238+0000 7fdd96be3640 1 -- 172.21.15.53:0/2954602996 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd90106c50 con 0x7fdd90100510 2024-09-19T06:10:51.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.238+0000 7fdd8dffb640 1 -- 172.21.15.53:0/2954602996 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdd8406d200 con 0x7fdd90100510 2024-09-19T06:10:51.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.238+0000 7fdd8ffff640 1 --2- 172.21.15.53:0/2954602996 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fdd80039380 0x7fdd8003b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:51.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.239+0000 7fdd8ffff640 1 --2- 172.21.15.53:0/2954602996 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fdd80039380 0x7fdd8003b840 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c0098c0 tx=0x7fdd7c006db0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:51.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.243+0000 7fdd8dffb640 1 -- 172.21.15.53:0/2954602996 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdd84040260 con 0x7fdd90100510 2024-09-19T06:10:51.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.446+0000 7fdd96be3640 1 -- 172.21.15.53:0/2954602996 --> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdd9002d990 con 0x7fdd80039380 2024-09-19T06:10:51.449 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:51 smithi053 ceph-mon[27968]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:10:51.449 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:51 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:51.449 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:51 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:51.449 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:51 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:51.449 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:51 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:51.449 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:51 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:51.449 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:51 smithi053 ceph-mon[27968]: Deploying daemon alertmanager.smithi053 on smithi053 2024-09-19T06:10:51.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.448+0000 7fdd8dffb640 1 -- 172.21.15.53:0/2954602996 <== mgr.14162 v2:172.21.15.53:6800/3684090301 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7fdd9002d990 con 0x7fdd80039380 2024-09-19T06:10:51.450 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:10:51.450 INFO:teuthology.orchestra.run.smithi053.stdout:[{"addr": "172.21.15.53", "hostname": "smithi053", "labels": [], "status": ""}, {"addr": "172.21.15.155", "hostname": "smithi155", "labels": [], "status": ""}] 2024-09-19T06:10:51.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.450+0000 7fdd96be3640 1 -- 172.21.15.53:0/2954602996 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fdd80039380 msgr2=0x7fdd8003b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:51.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.450+0000 7fdd96be3640 1 --2- 172.21.15.53:0/2954602996 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fdd80039380 0x7fdd8003b840 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c0098c0 tx=0x7fdd7c006db0 comp rx=0 tx=0).stop 2024-09-19T06:10:51.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.450+0000 7fdd96be3640 1 -- 172.21.15.53:0/2954602996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd90100510 msgr2=0x7fdd9006a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:51.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.450+0000 7fdd96be3640 1 --2- 172.21.15.53:0/2954602996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd90100510 0x7fdd9006a950 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdd840046e0 tx=0x7fdd84004710 comp rx=0 tx=0).stop 2024-09-19T06:10:51.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.450+0000 7fdd96be3640 1 -- 172.21.15.53:0/2954602996 shutdown_connections 2024-09-19T06:10:51.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.450+0000 7fdd96be3640 1 --2- 172.21.15.53:0/2954602996 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fdd80039380 0x7fdd8003b840 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:51.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.450+0000 7fdd96be3640 1 --2- 172.21.15.53:0/2954602996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd90100510 0x7fdd9006a950 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:51.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.450+0000 7fdd96be3640 1 -- 172.21.15.53:0/2954602996 >> 172.21.15.53:0/2954602996 conn(0x7fdd900fbc80 msgr2=0x7fdd900fc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:51.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.450+0000 7fdd96be3640 1 -- 172.21.15.53:0/2954602996 shutdown_connections 2024-09-19T06:10:51.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:51.450+0000 7fdd96be3640 1 -- 172.21.15.53:0/2954602996 wait complete. 2024-09-19T06:10:52.120 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-19T06:10:52.120 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-19T06:10:52.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:52 smithi053 ceph-mon[27968]: from='client.14189 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:10:52.480 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:10:53.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:53.269+0000 7f22e2b90640 1 -- 172.21.15.53:0/3460782624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22dc101600 msgr2=0x7f22dc101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:53.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:53.269+0000 7f22e2b90640 1 --2- 172.21.15.53:0/3460782624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22dc101600 0x7f22dc101a00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009880 tx=0x7f22d002a5d0 comp rx=0 tx=0).stop 2024-09-19T06:10:53.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:53.269+0000 7f22e2b90640 1 -- 172.21.15.53:0/3460782624 shutdown_connections 2024-09-19T06:10:53.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:53.269+0000 7f22e2b90640 1 --2- 172.21.15.53:0/3460782624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22dc101600 0x7f22dc101a00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:53.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:53.269+0000 7f22e2b90640 1 -- 172.21.15.53:0/3460782624 >> 172.21.15.53:0/3460782624 conn(0x7f22dc0fcdb0 msgr2=0x7f22dc0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:53.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:53.269+0000 7f22e2b90640 1 -- 172.21.15.53:0/3460782624 shutdown_connections 2024-09-19T06:10:53.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:53.269+0000 7f22e2b90640 1 -- 172.21.15.53:0/3460782624 wait complete. 2024-09-19T06:10:53.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:53.269+0000 7f22e2b90640 1 Processor -- start 2024-09-19T06:10:53.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:53.269+0000 7f22e2b90640 1 -- start start 2024-09-19T06:10:53.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:53.270+0000 7f22e2b90640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22dc101600 0x7f22dc198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:53.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:53.270+0000 7f22e2b90640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc1075c0 con 0x7f22dc101600 2024-09-19T06:10:53.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:53.270+0000 7f22e092c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22dc101600 0x7f22dc198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:53.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:53.270+0000 7f22e092c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22dc101600 0x7f22dc198ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41240/0 (socket says 172.21.15.53:41240) 2024-09-19T06:10:53.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:53.270+0000 7f22e092c640 1 -- 172.21.15.53:0/1212914618 learned_addr learned my addr 172.21.15.53:0/1212914618 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:10:53.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:53.270+0000 7f22e092c640 1 -- 172.21.15.53:0/1212914618 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22dc199430 con 0x7f22dc101600 2024-09-19T06:10:53.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:53.270+0000 7f22e092c640 1 --2- 172.21.15.53:0/1212914618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22dc101600 0x7f22dc198ef0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f22d0005ab0 tx=0x7f22d00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:53.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:53.270+0000 7f22d9ffb640 1 -- 172.21.15.53:0/1212914618 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f22d003f070 con 0x7f22dc101600 2024-09-19T06:10:53.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:53.270+0000 7f22d9ffb640 1 -- 172.21.15.53:0/1212914618 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22d0003aa0 con 0x7f22dc101600 2024-09-19T06:10:53.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:53.270+0000 7f22e2b90640 1 -- 172.21.15.53:0/1212914618 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22dc1996c0 con 0x7f22dc101600 2024-09-19T06:10:53.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:53.270+0000 7f22d9ffb640 1 -- 172.21.15.53:0/1212914618 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f22d0035030 con 0x7f22dc101600 2024-09-19T06:10:53.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:53.271+0000 7f22e2b90640 1 -- 172.21.15.53:0/1212914618 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22dc19c3b0 con 0x7f22dc101600 2024-09-19T06:10:53.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:53.271+0000 7f22d9ffb640 1 -- 172.21.15.53:0/1212914618 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46532+0+0 (secure 0 0 0) 0x7f22d0042070 con 0x7f22dc101600 2024-09-19T06:10:53.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:53.272+0000 7f22e2b90640 1 -- 172.21.15.53:0/1212914618 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22dc107d40 con 0x7f22dc101600 2024-09-19T06:10:53.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:53.272+0000 7f22d9ffb640 1 --2- 172.21.15.53:0/1212914618 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f22cc039330 0x7f22cc03b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:53.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:53.272+0000 7f22d9ffb640 1 -- 172.21.15.53:0/1212914618 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f22d006cff0 con 0x7f22dc101600 2024-09-19T06:10:53.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:53.272+0000 7f22dbfff640 1 --2- 172.21.15.53:0/1212914618 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f22cc039330 0x7f22cc03b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:53.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:53.272+0000 7f22dbfff640 1 --2- 172.21.15.53:0/1212914618 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f22cc039330 0x7f22cc03b7f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f22c80098c0 tx=0x7f22c8006db0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:53.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:53.277+0000 7f22d9ffb640 1 -- 172.21.15.53:0/1212914618 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f22d0040260 con 0x7f22dc101600 2024-09-19T06:10:53.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:53.457+0000 7f22e2b90640 1 -- 172.21.15.53:0/1212914618 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f22dc02d7c0 con 0x7f22dc101600 2024-09-19T06:10:53.460 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:53 smithi053 ceph-mon[27968]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:10:54.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:54.253+0000 7f22d9ffb640 1 -- 172.21.15.53:0/1212914618 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f22d004cc30 con 0x7f22dc101600 2024-09-19T06:10:54.256 INFO:teuthology.orchestra.run.smithi053.stderr:adjusted tunables profile to default 2024-09-19T06:10:54.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:54.256+0000 7f22e2b90640 1 -- 172.21.15.53:0/1212914618 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f22cc039330 msgr2=0x7f22cc03b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:54.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:54.256+0000 7f22e2b90640 1 --2- 172.21.15.53:0/1212914618 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f22cc039330 0x7f22cc03b7f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f22c80098c0 tx=0x7f22c8006db0 comp rx=0 tx=0).stop 2024-09-19T06:10:54.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:54.256+0000 7f22e2b90640 1 -- 172.21.15.53:0/1212914618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22dc101600 msgr2=0x7f22dc198ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:54.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:54.256+0000 7f22e2b90640 1 --2- 172.21.15.53:0/1212914618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22dc101600 0x7f22dc198ef0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f22d0005ab0 tx=0x7f22d00048f0 comp rx=0 tx=0).stop 2024-09-19T06:10:54.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:54.256+0000 7f22e2b90640 1 -- 172.21.15.53:0/1212914618 shutdown_connections 2024-09-19T06:10:54.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:54.256+0000 7f22e2b90640 1 --2- 172.21.15.53:0/1212914618 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f22cc039330 0x7f22cc03b7f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:54.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:54.256+0000 7f22e2b90640 1 --2- 172.21.15.53:0/1212914618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22dc101600 0x7f22dc198ef0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:54.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:54.256+0000 7f22e2b90640 1 -- 172.21.15.53:0/1212914618 >> 172.21.15.53:0/1212914618 conn(0x7f22dc0fcdb0 msgr2=0x7f22dc0ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:54.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:54.256+0000 7f22e2b90640 1 -- 172.21.15.53:0/1212914618 shutdown_connections 2024-09-19T06:10:54.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:10:54.256+0000 7f22e2b90640 1 -- 172.21.15.53:0/1212914618 wait complete. 2024-09-19T06:10:54.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:54 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/1212914618' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T06:10:55.546 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:55 smithi053 ceph-mon[27968]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:10:55.546 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:55 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/1212914618' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T06:10:55.546 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:55 smithi053 ceph-mon[27968]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T06:10:55.546 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:55 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:55.573 INFO:tasks.cephadm:Adding mon.smithi053 on smithi053 2024-09-19T06:10:55.574 INFO:tasks.cephadm:Adding mon.smithi155 on smithi155 2024-09-19T06:10:55.574 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph orch apply mon '2;smithi053:172.21.15.53=smithi053;smithi155:172.21.15.155=smithi155' 2024-09-19T06:10:55.819 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:10:55.887 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:10:56.284 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.281+0000 7f64da685640 1 -- 172.21.15.155:0/2566876856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64d41020a0 msgr2=0x7f64d4104490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:56.284 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.281+0000 7f64da685640 1 --2- 172.21.15.155:0/2566876856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64d41020a0 0x7f64d4104490 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f64c4009880 tx=0x7f64c402a5d0 comp rx=0 tx=0).stop 2024-09-19T06:10:56.285 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.281+0000 7f64da685640 1 -- 172.21.15.155:0/2566876856 shutdown_connections 2024-09-19T06:10:56.285 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.281+0000 7f64da685640 1 --2- 172.21.15.155:0/2566876856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64d41020a0 0x7f64d4104490 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:56.285 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.281+0000 7f64da685640 1 -- 172.21.15.155:0/2566876856 >> 172.21.15.155:0/2566876856 conn(0x7f64d40fbc80 msgr2=0x7f64d40fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:56.285 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.281+0000 7f64da685640 1 -- 172.21.15.155:0/2566876856 shutdown_connections 2024-09-19T06:10:56.285 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.281+0000 7f64da685640 1 -- 172.21.15.155:0/2566876856 wait complete. 2024-09-19T06:10:56.285 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.282+0000 7f64da685640 1 Processor -- start 2024-09-19T06:10:56.285 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.282+0000 7f64da685640 1 -- start start 2024-09-19T06:10:56.285 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.282+0000 7f64da685640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64d41020a0 0x7f64d4101920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:56.285 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.282+0000 7f64da685640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f64d4107560 con 0x7f64d41020a0 2024-09-19T06:10:56.285 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.282+0000 7f64d3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64d41020a0 0x7f64d4101920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:56.285 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.282+0000 7f64d3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64d41020a0 0x7f64d4101920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.155:39262/0 (socket says 172.21.15.155:39262) 2024-09-19T06:10:56.285 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.282+0000 7f64d3fff640 1 -- 172.21.15.155:0/1076308860 learned_addr learned my addr 172.21.15.155:0/1076308860 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-19T06:10:56.285 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.282+0000 7f64d3fff640 1 -- 172.21.15.155:0/1076308860 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64d4101e60 con 0x7f64d41020a0 2024-09-19T06:10:56.286 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.283+0000 7f64d3fff640 1 --2- 172.21.15.155:0/1076308860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64d41020a0 0x7f64d4101920 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f64c4006010 tx=0x7f64c40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:56.286 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.283+0000 7f64d1ffb640 1 -- 172.21.15.155:0/1076308860 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f64c403f070 con 0x7f64d41020a0 2024-09-19T06:10:56.286 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.283+0000 7f64da685640 1 -- 172.21.15.155:0/1076308860 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64d4100050 con 0x7f64d41020a0 2024-09-19T06:10:56.286 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.283+0000 7f64d1ffb640 1 -- 172.21.15.155:0/1076308860 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64c4003aa0 con 0x7f64d41020a0 2024-09-19T06:10:56.286 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.283+0000 7f64d1ffb640 1 -- 172.21.15.155:0/1076308860 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f64c4035030 con 0x7f64d41020a0 2024-09-19T06:10:56.286 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.283+0000 7f64da685640 1 -- 172.21.15.155:0/1076308860 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64d4100590 con 0x7f64d41020a0 2024-09-19T06:10:56.287 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.284+0000 7f64d1ffb640 1 -- 172.21.15.155:0/1076308860 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46532+0+0 (secure 0 0 0) 0x7f64c4042070 con 0x7f64d41020a0 2024-09-19T06:10:56.287 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.284+0000 7f64da685640 1 -- 172.21.15.155:0/1076308860 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64d4107ce0 con 0x7f64d41020a0 2024-09-19T06:10:56.287 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.284+0000 7f64d1ffb640 1 --2- 172.21.15.155:0/1076308860 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f64c0039330 0x7f64c003b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:56.287 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.284+0000 7f64d1ffb640 1 -- 172.21.15.155:0/1076308860 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f64c406cfa0 con 0x7f64d41020a0 2024-09-19T06:10:56.287 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.284+0000 7f64cbfff640 1 --2- 172.21.15.155:0/1076308860 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f64c0039330 0x7f64c003b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:56.287 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.285+0000 7f64cbfff640 1 --2- 172.21.15.155:0/1076308860 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f64c0039330 0x7f64c003b7f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f64bc0098c0 tx=0x7f64bc006db0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:56.291 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.289+0000 7f64d1ffb640 1 -- 172.21.15.155:0/1076308860 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f64c4040260 con 0x7f64d41020a0 2024-09-19T06:10:56.465 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.463+0000 7f64da685640 1 -- 172.21.15.155:0/1076308860 --> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi053:172.21.15.53=smithi053;smithi155:172.21.15.155=smithi155", "target": ["mon-mgr", ""]}) -- 0x7f64d41902f0 con 0x7f64c0039330 2024-09-19T06:10:56.470 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.468+0000 7f64d1ffb640 1 -- 172.21.15.155:0/1076308860 <== mgr.14162 v2:172.21.15.53:6800/3684090301 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f64d41902f0 con 0x7f64c0039330 2024-09-19T06:10:56.470 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled mon update... 2024-09-19T06:10:56.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.470+0000 7f64da685640 1 -- 172.21.15.155:0/1076308860 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f64c0039330 msgr2=0x7f64c003b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:56.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.470+0000 7f64da685640 1 --2- 172.21.15.155:0/1076308860 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f64c0039330 0x7f64c003b7f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f64bc0098c0 tx=0x7f64bc006db0 comp rx=0 tx=0).stop 2024-09-19T06:10:56.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.470+0000 7f64da685640 1 -- 172.21.15.155:0/1076308860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64d41020a0 msgr2=0x7f64d4101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:56.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.470+0000 7f64da685640 1 --2- 172.21.15.155:0/1076308860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64d41020a0 0x7f64d4101920 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f64c4006010 tx=0x7f64c40048f0 comp rx=0 tx=0).stop 2024-09-19T06:10:56.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.470+0000 7f64da685640 1 -- 172.21.15.155:0/1076308860 shutdown_connections 2024-09-19T06:10:56.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.470+0000 7f64da685640 1 --2- 172.21.15.155:0/1076308860 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f64c0039330 0x7f64c003b7f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:56.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.470+0000 7f64da685640 1 --2- 172.21.15.155:0/1076308860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64d41020a0 0x7f64d4101920 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:56.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.470+0000 7f64da685640 1 -- 172.21.15.155:0/1076308860 >> 172.21.15.155:0/1076308860 conn(0x7f64d40fbc80 msgr2=0x7f64d40fc770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:56.473 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.470+0000 7f64da685640 1 -- 172.21.15.155:0/1076308860 shutdown_connections 2024-09-19T06:10:56.473 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:56.470+0000 7f64da685640 1 -- 172.21.15.155:0/1076308860 wait complete. 2024-09-19T06:10:57.041 DEBUG:teuthology.orchestra.run.smithi155:mon.smithi155> sudo journalctl -f -n 0 -u ceph-9f2876d0-764d-11ef-bceb-c7b262605968@mon.smithi155.service 2024-09-19T06:10:57.043 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-19T06:10:57.043 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T06:10:57.311 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:10:57.355 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:10:57.536 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:57 smithi053 ceph-mon[27968]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:10:57.536 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:57 smithi053 ceph-mon[27968]: from='client.14193 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi053:172.21.15.53=smithi053;smithi155:172.21.15.155=smithi155", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:10:57.536 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:57 smithi053 ceph-mon[27968]: Saving service mon spec with placement smithi053:172.21.15.53=smithi053;smithi155:172.21.15.155=smithi155;count:2 2024-09-19T06:10:57.536 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:57 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:57.765 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.758+0000 7fc75abb9640 1 -- 172.21.15.155:0/909498565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc754100510 msgr2=0x7fc754100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:57.765 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.758+0000 7fc75abb9640 1 --2- 172.21.15.155:0/909498565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc754100510 0x7fc754100910 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc748009880 tx=0x7fc74802a5d0 comp rx=0 tx=0).stop 2024-09-19T06:10:57.765 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.758+0000 7fc75abb9640 1 -- 172.21.15.155:0/909498565 shutdown_connections 2024-09-19T06:10:57.765 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.758+0000 7fc75abb9640 1 --2- 172.21.15.155:0/909498565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc754100510 0x7fc754100910 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:57.765 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.758+0000 7fc75abb9640 1 -- 172.21.15.155:0/909498565 >> 172.21.15.155:0/909498565 conn(0x7fc7540fbc80 msgr2=0x7fc7540fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:57.766 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.758+0000 7fc75abb9640 1 -- 172.21.15.155:0/909498565 shutdown_connections 2024-09-19T06:10:57.766 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.758+0000 7fc75abb9640 1 -- 172.21.15.155:0/909498565 wait complete. 2024-09-19T06:10:57.766 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.758+0000 7fc75abb9640 1 Processor -- start 2024-09-19T06:10:57.766 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.758+0000 7fc75abb9640 1 -- start start 2024-09-19T06:10:57.766 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.759+0000 7fc75abb9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc754100510 0x7fc754195be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:57.766 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.759+0000 7fc75abb9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc7541064d0 con 0x7fc754100510 2024-09-19T06:10:57.766 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.759+0000 7fc758955640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc754100510 0x7fc754195be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:57.766 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.759+0000 7fc758955640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc754100510 0x7fc754195be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.155:39282/0 (socket says 172.21.15.155:39282) 2024-09-19T06:10:57.766 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.759+0000 7fc758955640 1 -- 172.21.15.155:0/997975947 learned_addr learned my addr 172.21.15.155:0/997975947 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-19T06:10:57.766 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.759+0000 7fc758955640 1 -- 172.21.15.155:0/997975947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc754196120 con 0x7fc754100510 2024-09-19T06:10:57.766 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.759+0000 7fc758955640 1 --2- 172.21.15.155:0/997975947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc754100510 0x7fc754195be0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc748005ab0 tx=0x7fc7480048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:57.766 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.759+0000 7fc751ffb640 1 -- 172.21.15.155:0/997975947 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fc74803f070 con 0x7fc754100510 2024-09-19T06:10:57.767 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.759+0000 7fc751ffb640 1 -- 172.21.15.155:0/997975947 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc748003aa0 con 0x7fc754100510 2024-09-19T06:10:57.767 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.759+0000 7fc75abb9640 1 -- 172.21.15.155:0/997975947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7541963b0 con 0x7fc754100510 2024-09-19T06:10:57.767 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.759+0000 7fc751ffb640 1 -- 172.21.15.155:0/997975947 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fc748035030 con 0x7fc754100510 2024-09-19T06:10:57.767 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.760+0000 7fc75abb9640 1 -- 172.21.15.155:0/997975947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7541990a0 con 0x7fc754100510 2024-09-19T06:10:57.767 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.760+0000 7fc751ffb640 1 -- 172.21.15.155:0/997975947 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46532+0+0 (secure 0 0 0) 0x7fc748042070 con 0x7fc754100510 2024-09-19T06:10:57.767 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.761+0000 7fc75abb9640 1 -- 172.21.15.155:0/997975947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc754106c50 con 0x7fc754100510 2024-09-19T06:10:57.767 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.761+0000 7fc751ffb640 1 --2- 172.21.15.155:0/997975947 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fc744039330 0x7fc74403b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:57.767 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.761+0000 7fc751ffb640 1 -- 172.21.15.155:0/997975947 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc74806d0b0 con 0x7fc754100510 2024-09-19T06:10:57.767 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.761+0000 7fc753fff640 1 --2- 172.21.15.155:0/997975947 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fc744039330 0x7fc74403b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:57.767 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.762+0000 7fc753fff640 1 --2- 172.21.15.155:0/997975947 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fc744039330 0x7fc74403b7f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc7400098c0 tx=0x7fc740006db0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:57.767 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.766+0000 7fc751ffb640 1 -- 172.21.15.155:0/997975947 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc748040260 con 0x7fc754100510 2024-09-19T06:10:57.995 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.993+0000 7fc75abb9640 1 -- 172.21.15.155:0/997975947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc75402d7c0 con 0x7fc754100510 2024-09-19T06:10:57.995 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.994+0000 7fc751ffb640 1 -- 172.21.15.155:0/997975947 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+753 (secure 0 0 0) 0x7fc74804cc30 con 0x7fc754100510 2024-09-19T06:10:57.996 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:10:57.996 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":1,"fsid":"9f2876d0-764d-11ef-bceb-c7b262605968","modified":"2024-09-19T06:09:23.791881Z","created":"2024-09-19T06:09:23.791881Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi053","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T06:10:57.996 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 1 2024-09-19T06:10:57.998 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.996+0000 7fc75abb9640 1 -- 172.21.15.155:0/997975947 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fc744039330 msgr2=0x7fc74403b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:57.998 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.996+0000 7fc75abb9640 1 --2- 172.21.15.155:0/997975947 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fc744039330 0x7fc74403b7f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc7400098c0 tx=0x7fc740006db0 comp rx=0 tx=0).stop 2024-09-19T06:10:57.998 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.996+0000 7fc75abb9640 1 -- 172.21.15.155:0/997975947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc754100510 msgr2=0x7fc754195be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:57.998 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.996+0000 7fc75abb9640 1 --2- 172.21.15.155:0/997975947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc754100510 0x7fc754195be0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc748005ab0 tx=0x7fc7480048f0 comp rx=0 tx=0).stop 2024-09-19T06:10:57.998 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.996+0000 7fc75abb9640 1 -- 172.21.15.155:0/997975947 shutdown_connections 2024-09-19T06:10:57.998 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.996+0000 7fc75abb9640 1 --2- 172.21.15.155:0/997975947 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fc744039330 0x7fc74403b7f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:57.998 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.996+0000 7fc75abb9640 1 --2- 172.21.15.155:0/997975947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc754100510 0x7fc754195be0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:57.998 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.996+0000 7fc75abb9640 1 -- 172.21.15.155:0/997975947 >> 172.21.15.155:0/997975947 conn(0x7fc7540fbc80 msgr2=0x7fc7540fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:57.998 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.996+0000 7fc75abb9640 1 -- 172.21.15.155:0/997975947 shutdown_connections 2024-09-19T06:10:57.998 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:10:57.996+0000 7fc75abb9640 1 -- 172.21.15.155:0/997975947 wait complete. 2024-09-19T06:10:58.850 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:58 smithi053 ceph-mon[27968]: from='client.? 172.21.15.155:0/997975947' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:10:59.614 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-19T06:10:59.615 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T06:10:59.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:59 smithi053 ceph-mon[27968]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:10:59.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:59 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:59.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:59 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:59.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:59 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:59.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:59 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:59.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:59 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:59.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:59 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:59.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:59 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:59.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:59 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:59.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:59 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-19T06:10:59.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:10:59 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:10:59.860 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:10:59.904 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:11:00.297 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.278+0000 7f9ea9e11640 1 -- 172.21.15.155:0/4114018829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ea40ffe90 msgr2=0x7f9ea41022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:00.297 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.278+0000 7f9ea9e11640 1 --2- 172.21.15.155:0/4114018829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ea40ffe90 0x7f9ea41022a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009880 tx=0x7f9e9402a5d0 comp rx=0 tx=0).stop 2024-09-19T06:11:00.297 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.278+0000 7f9ea9e11640 1 -- 172.21.15.155:0/4114018829 shutdown_connections 2024-09-19T06:11:00.297 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.278+0000 7f9ea9e11640 1 --2- 172.21.15.155:0/4114018829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ea40ffe90 0x7f9ea41022a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:00.297 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.278+0000 7f9ea9e11640 1 -- 172.21.15.155:0/4114018829 >> 172.21.15.155:0/4114018829 conn(0x7f9ea40fbc80 msgr2=0x7f9ea40fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:00.297 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.278+0000 7f9ea9e11640 1 -- 172.21.15.155:0/4114018829 shutdown_connections 2024-09-19T06:11:00.297 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.278+0000 7f9ea9e11640 1 -- 172.21.15.155:0/4114018829 wait complete. 2024-09-19T06:11:00.297 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.279+0000 7f9ea9e11640 1 Processor -- start 2024-09-19T06:11:00.297 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.279+0000 7f9ea9e11640 1 -- start start 2024-09-19T06:11:00.297 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.279+0000 7f9ea9e11640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ea40ffe90 0x7f9ea419d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:00.297 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.279+0000 7f9ea9e11640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea4105480 con 0x7f9ea40ffe90 2024-09-19T06:11:00.298 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.279+0000 7f9ea37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ea40ffe90 0x7f9ea419d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:00.298 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.279+0000 7f9ea37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ea40ffe90 0x7f9ea419d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.155:35374/0 (socket says 172.21.15.155:35374) 2024-09-19T06:11:00.298 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.279+0000 7f9ea37fe640 1 -- 172.21.15.155:0/703382073 learned_addr learned my addr 172.21.15.155:0/703382073 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-19T06:11:00.298 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.279+0000 7f9ea37fe640 1 -- 172.21.15.155:0/703382073 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea419d7b0 con 0x7f9ea40ffe90 2024-09-19T06:11:00.298 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.280+0000 7f9ea37fe640 1 --2- 172.21.15.155:0/703382073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ea40ffe90 0x7f9ea419d270 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9e94005ab0 tx=0x7f9e940048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:00.298 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.280+0000 7f9ea0ff9640 1 -- 172.21.15.155:0/703382073 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f9e9403f070 con 0x7f9ea40ffe90 2024-09-19T06:11:00.298 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.280+0000 7f9ea9e11640 1 -- 172.21.15.155:0/703382073 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ea419da40 con 0x7f9ea40ffe90 2024-09-19T06:11:00.298 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.280+0000 7f9ea0ff9640 1 -- 172.21.15.155:0/703382073 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e94003aa0 con 0x7f9ea40ffe90 2024-09-19T06:11:00.298 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.280+0000 7f9ea0ff9640 1 -- 172.21.15.155:0/703382073 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f9e94035030 con 0x7f9ea40ffe90 2024-09-19T06:11:00.298 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.280+0000 7f9ea9e11640 1 -- 172.21.15.155:0/703382073 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ea41a0730 con 0x7f9ea40ffe90 2024-09-19T06:11:00.298 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.281+0000 7f9ea0ff9640 1 -- 172.21.15.155:0/703382073 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46532+0+0 (secure 0 0 0) 0x7f9e94042070 con 0x7f9ea40ffe90 2024-09-19T06:11:00.298 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.281+0000 7f9ea9e11640 1 -- 172.21.15.155:0/703382073 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ea4105c00 con 0x7f9ea40ffe90 2024-09-19T06:11:00.298 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.281+0000 7f9ea0ff9640 1 --2- 172.21.15.155:0/703382073 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f9e8c0392d0 0x7f9e8c03b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:00.298 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.281+0000 7f9ea0ff9640 1 -- 172.21.15.155:0/703382073 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9e9406d3d0 con 0x7f9ea40ffe90 2024-09-19T06:11:00.299 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.281+0000 7f9ea2ffd640 1 --2- 172.21.15.155:0/703382073 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f9e8c0392d0 0x7f9e8c03b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:00.299 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.282+0000 7f9ea2ffd640 1 --2- 172.21.15.155:0/703382073 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f9e8c0392d0 0x7f9e8c03b790 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9e980098c0 tx=0x7f9e98006db0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:00.299 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.286+0000 7f9ea0ff9640 1 -- 172.21.15.155:0/703382073 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9e940433b0 con 0x7f9ea40ffe90 2024-09-19T06:11:00.516 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.515+0000 7f9ea9e11640 1 -- 172.21.15.155:0/703382073 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9ea402d7c0 con 0x7f9ea40ffe90 2024-09-19T06:11:00.517 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.515+0000 7f9ea0ff9640 1 -- 172.21.15.155:0/703382073 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+753 (secure 0 0 0) 0x7f9e9402f300 con 0x7f9ea40ffe90 2024-09-19T06:11:00.517 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:11:00.517 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":1,"fsid":"9f2876d0-764d-11ef-bceb-c7b262605968","modified":"2024-09-19T06:09:23.791881Z","created":"2024-09-19T06:09:23.791881Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi053","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T06:11:00.517 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 1 2024-09-19T06:11:00.519 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.517+0000 7f9ea9e11640 1 -- 172.21.15.155:0/703382073 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f9e8c0392d0 msgr2=0x7f9e8c03b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:00.519 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.517+0000 7f9ea9e11640 1 --2- 172.21.15.155:0/703382073 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f9e8c0392d0 0x7f9e8c03b790 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9e980098c0 tx=0x7f9e98006db0 comp rx=0 tx=0).stop 2024-09-19T06:11:00.519 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.517+0000 7f9ea9e11640 1 -- 172.21.15.155:0/703382073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ea40ffe90 msgr2=0x7f9ea419d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:00.519 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.517+0000 7f9ea9e11640 1 --2- 172.21.15.155:0/703382073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ea40ffe90 0x7f9ea419d270 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9e94005ab0 tx=0x7f9e940048f0 comp rx=0 tx=0).stop 2024-09-19T06:11:00.519 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.517+0000 7f9ea9e11640 1 -- 172.21.15.155:0/703382073 shutdown_connections 2024-09-19T06:11:00.519 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.517+0000 7f9ea9e11640 1 --2- 172.21.15.155:0/703382073 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f9e8c0392d0 0x7f9e8c03b790 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:00.519 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.517+0000 7f9ea9e11640 1 --2- 172.21.15.155:0/703382073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ea40ffe90 0x7f9ea419d270 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:00.519 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.517+0000 7f9ea9e11640 1 -- 172.21.15.155:0/703382073 >> 172.21.15.155:0/703382073 conn(0x7f9ea40fbc80 msgr2=0x7f9ea40fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:00.519 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.517+0000 7f9ea9e11640 1 -- 172.21.15.155:0/703382073 shutdown_connections 2024-09-19T06:11:00.520 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:00.517+0000 7f9ea9e11640 1 -- 172.21.15.155:0/703382073 wait complete. 2024-09-19T06:11:00.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:00 smithi053 ceph-mon[27968]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-19T06:11:00.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:00 smithi053 ceph-mon[27968]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-19T06:11:00.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:00 smithi053 ceph-mon[27968]: Deploying daemon grafana.smithi053 on smithi053 2024-09-19T06:11:00.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:00 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:01.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:01 smithi053 ceph-mon[27968]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:01.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:01 smithi053 ceph-mon[27968]: from='client.? 172.21.15.155:0/703382073' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:02.219 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-19T06:11:02.219 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T06:11:02.476 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:11:02.520 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:11:02.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:02.903+0000 7f1d13c08640 1 -- 172.21.15.155:0/458184223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d0c1004e0 msgr2=0x7f1d0c1008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:02.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:02.903+0000 7f1d13c08640 1 --2- 172.21.15.155:0/458184223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d0c1004e0 0x7f1d0c1008e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1d08009880 tx=0x7f1d0802a5d0 comp rx=0 tx=0).stop 2024-09-19T06:11:02.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:02.903+0000 7f1d13c08640 1 -- 172.21.15.155:0/458184223 shutdown_connections 2024-09-19T06:11:02.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:02.903+0000 7f1d13c08640 1 --2- 172.21.15.155:0/458184223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d0c1004e0 0x7f1d0c1008e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:02.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:02.903+0000 7f1d13c08640 1 -- 172.21.15.155:0/458184223 >> 172.21.15.155:0/458184223 conn(0x7f1d0c0fbcb0 msgr2=0x7f1d0c0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:02.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:02.903+0000 7f1d13c08640 1 -- 172.21.15.155:0/458184223 shutdown_connections 2024-09-19T06:11:02.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:02.903+0000 7f1d13c08640 1 -- 172.21.15.155:0/458184223 wait complete. 2024-09-19T06:11:02.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:02.903+0000 7f1d13c08640 1 Processor -- start 2024-09-19T06:11:02.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:02.903+0000 7f1d13c08640 1 -- start start 2024-09-19T06:11:02.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:02.904+0000 7f1d13c08640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d0c1004e0 0x7f1d0c194ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:02.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:02.904+0000 7f1d13c08640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0c1053a0 con 0x7f1d0c1004e0 2024-09-19T06:11:02.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:02.904+0000 7f1d119a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d0c1004e0 0x7f1d0c194ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:02.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:02.904+0000 7f1d119a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d0c1004e0 0x7f1d0c194ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.155:35400/0 (socket says 172.21.15.155:35400) 2024-09-19T06:11:02.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:02.904+0000 7f1d119a4640 1 -- 172.21.15.155:0/2254246009 learned_addr learned my addr 172.21.15.155:0/2254246009 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-19T06:11:02.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:02.904+0000 7f1d119a4640 1 -- 172.21.15.155:0/2254246009 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d0c195010 con 0x7f1d0c1004e0 2024-09-19T06:11:02.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:02.904+0000 7f1d119a4640 1 --2- 172.21.15.155:0/2254246009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d0c1004e0 0x7f1d0c194ad0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1d080046e0 tx=0x7f1d08004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:02.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:02.905+0000 7f1cfeffd640 1 -- 172.21.15.155:0/2254246009 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f1d0803f070 con 0x7f1d0c1004e0 2024-09-19T06:11:02.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:02.905+0000 7f1cfeffd640 1 -- 172.21.15.155:0/2254246009 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d08003aa0 con 0x7f1d0c1004e0 2024-09-19T06:11:02.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:02.905+0000 7f1cfeffd640 1 -- 172.21.15.155:0/2254246009 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f1d08035030 con 0x7f1d0c1004e0 2024-09-19T06:11:02.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:02.905+0000 7f1d13c08640 1 -- 172.21.15.155:0/2254246009 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d0c1952a0 con 0x7f1d0c1004e0 2024-09-19T06:11:02.912 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:02.905+0000 7f1d13c08640 1 -- 172.21.15.155:0/2254246009 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d0c197f90 con 0x7f1d0c1004e0 2024-09-19T06:11:02.912 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:02.905+0000 7f1d13c08640 1 -- 172.21.15.155:0/2254246009 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d0c105b20 con 0x7f1d0c1004e0 2024-09-19T06:11:02.912 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:02.906+0000 7f1cfeffd640 1 -- 172.21.15.155:0/2254246009 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46532+0+0 (secure 0 0 0) 0x7f1d08042070 con 0x7f1d0c1004e0 2024-09-19T06:11:02.912 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:02.906+0000 7f1cfeffd640 1 --2- 172.21.15.155:0/2254246009 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f1cf803b520 0x7f1cf803d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:02.912 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:02.906+0000 7f1cfeffd640 1 -- 172.21.15.155:0/2254246009 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1d0806d160 con 0x7f1d0c1004e0 2024-09-19T06:11:02.912 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:02.906+0000 7f1d111a3640 1 --2- 172.21.15.155:0/2254246009 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f1cf803b520 0x7f1cf803d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:02.912 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:02.906+0000 7f1d111a3640 1 --2- 172.21.15.155:0/2254246009 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f1cf803b520 0x7f1cf803d9e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1d000098c0 tx=0x7f1d00006db0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:02.914 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:02.912+0000 7f1cfeffd640 1 -- 172.21.15.155:0/2254246009 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1d08040260 con 0x7f1d0c1004e0 2024-09-19T06:11:03.143 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:03.142+0000 7f1d13c08640 1 -- 172.21.15.155:0/2254246009 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f1d0c02d7c0 con 0x7f1d0c1004e0 2024-09-19T06:11:03.144 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:03.142+0000 7f1cfeffd640 1 -- 172.21.15.155:0/2254246009 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+753 (secure 0 0 0) 0x7f1d08041d00 con 0x7f1d0c1004e0 2024-09-19T06:11:03.144 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:11:03.144 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":1,"fsid":"9f2876d0-764d-11ef-bceb-c7b262605968","modified":"2024-09-19T06:09:23.791881Z","created":"2024-09-19T06:09:23.791881Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi053","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T06:11:03.145 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 1 2024-09-19T06:11:03.146 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:03.144+0000 7f1d13c08640 1 -- 172.21.15.155:0/2254246009 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f1cf803b520 msgr2=0x7f1cf803d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:03.146 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:03.144+0000 7f1d13c08640 1 --2- 172.21.15.155:0/2254246009 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f1cf803b520 0x7f1cf803d9e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1d000098c0 tx=0x7f1d00006db0 comp rx=0 tx=0).stop 2024-09-19T06:11:03.146 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:03.144+0000 7f1d13c08640 1 -- 172.21.15.155:0/2254246009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d0c1004e0 msgr2=0x7f1d0c194ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:03.146 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:03.144+0000 7f1d13c08640 1 --2- 172.21.15.155:0/2254246009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d0c1004e0 0x7f1d0c194ad0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1d080046e0 tx=0x7f1d08004710 comp rx=0 tx=0).stop 2024-09-19T06:11:03.146 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:03.144+0000 7f1d13c08640 1 -- 172.21.15.155:0/2254246009 shutdown_connections 2024-09-19T06:11:03.146 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:03.144+0000 7f1d13c08640 1 --2- 172.21.15.155:0/2254246009 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f1cf803b520 0x7f1cf803d9e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:03.146 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:03.144+0000 7f1d13c08640 1 --2- 172.21.15.155:0/2254246009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d0c1004e0 0x7f1d0c194ad0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:03.147 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:03.144+0000 7f1d13c08640 1 -- 172.21.15.155:0/2254246009 >> 172.21.15.155:0/2254246009 conn(0x7f1d0c0fbcb0 msgr2=0x7f1d0c0fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:03.147 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:03.144+0000 7f1d13c08640 1 -- 172.21.15.155:0/2254246009 shutdown_connections 2024-09-19T06:11:03.147 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:03.144+0000 7f1d13c08640 1 -- 172.21.15.155:0/2254246009 wait complete. 2024-09-19T06:11:03.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:03 smithi053 ceph-mon[27968]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:03.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:03 smithi053 ceph-mon[27968]: from='client.? 172.21.15.155:0/2254246009' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:04.692 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-19T06:11:04.692 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T06:11:04.933 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:11:04.977 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:11:05.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.436+0000 7f6652551640 1 -- 172.21.15.155:0/1466365859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f664c100510 msgr2=0x7f664c100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:05.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.436+0000 7f6652551640 1 --2- 172.21.15.155:0/1466365859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f664c100510 0x7f664c100910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f663c009880 tx=0x7f663c02a5d0 comp rx=0 tx=0).stop 2024-09-19T06:11:05.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.436+0000 7f6652551640 1 -- 172.21.15.155:0/1466365859 shutdown_connections 2024-09-19T06:11:05.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.436+0000 7f6652551640 1 --2- 172.21.15.155:0/1466365859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f664c100510 0x7f664c100910 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:05.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.436+0000 7f6652551640 1 -- 172.21.15.155:0/1466365859 >> 172.21.15.155:0/1466365859 conn(0x7f664c0fbc80 msgr2=0x7f664c0fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:05.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.436+0000 7f6652551640 1 -- 172.21.15.155:0/1466365859 shutdown_connections 2024-09-19T06:11:05.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.436+0000 7f6652551640 1 -- 172.21.15.155:0/1466365859 wait complete. 2024-09-19T06:11:05.442 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.436+0000 7f6652551640 1 Processor -- start 2024-09-19T06:11:05.442 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.437+0000 7f6652551640 1 -- start start 2024-09-19T06:11:05.442 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.437+0000 7f6652551640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f664c100510 0x7f664c195bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:05.442 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.437+0000 7f6652551640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f664c1064d0 con 0x7f664c100510 2024-09-19T06:11:05.442 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.437+0000 7f664bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f664c100510 0x7f664c195bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:05.442 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.437+0000 7f664bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f664c100510 0x7f664c195bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.155:35414/0 (socket says 172.21.15.155:35414) 2024-09-19T06:11:05.442 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.437+0000 7f664bfff640 1 -- 172.21.15.155:0/1743455497 learned_addr learned my addr 172.21.15.155:0/1743455497 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-19T06:11:05.442 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.437+0000 7f664bfff640 1 -- 172.21.15.155:0/1743455497 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f664c196110 con 0x7f664c100510 2024-09-19T06:11:05.442 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.437+0000 7f664bfff640 1 --2- 172.21.15.155:0/1743455497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f664c100510 0x7f664c195bd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f663c0046e0 tx=0x7f663c004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:05.442 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.438+0000 7f6649ffb640 1 -- 172.21.15.155:0/1743455497 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f663c03f070 con 0x7f664c100510 2024-09-19T06:11:05.442 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.438+0000 7f6649ffb640 1 -- 172.21.15.155:0/1743455497 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f663c003aa0 con 0x7f664c100510 2024-09-19T06:11:05.443 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.438+0000 7f6652551640 1 -- 172.21.15.155:0/1743455497 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f664c1963a0 con 0x7f664c100510 2024-09-19T06:11:05.443 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.438+0000 7f6649ffb640 1 -- 172.21.15.155:0/1743455497 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f663c035030 con 0x7f664c100510 2024-09-19T06:11:05.443 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.438+0000 7f6652551640 1 -- 172.21.15.155:0/1743455497 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f664c199090 con 0x7f664c100510 2024-09-19T06:11:05.443 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.438+0000 7f6649ffb640 1 -- 172.21.15.155:0/1743455497 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46532+0+0 (secure 0 0 0) 0x7f663c042070 con 0x7f664c100510 2024-09-19T06:11:05.443 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.439+0000 7f6652551640 1 -- 172.21.15.155:0/1743455497 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f664c106c50 con 0x7f664c100510 2024-09-19T06:11:05.443 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.439+0000 7f6649ffb640 1 --2- 172.21.15.155:0/1743455497 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f66380392d0 0x7f663803b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:05.443 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.439+0000 7f6649ffb640 1 -- 172.21.15.155:0/1743455497 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f663c06d220 con 0x7f664c100510 2024-09-19T06:11:05.443 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.439+0000 7f6643fff640 1 --2- 172.21.15.155:0/1743455497 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f66380392d0 0x7f663803b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:05.443 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.439+0000 7f6643fff640 1 --2- 172.21.15.155:0/1743455497 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f66380392d0 0x7f663803b790 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f66340098c0 tx=0x7f6634006db0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:05.445 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.444+0000 7f6649ffb640 1 -- 172.21.15.155:0/1743455497 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f663c039350 con 0x7f664c100510 2024-09-19T06:11:05.675 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.673+0000 7f6652551640 1 -- 172.21.15.155:0/1743455497 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f664c02d7c0 con 0x7f664c100510 2024-09-19T06:11:05.675 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.674+0000 7f6649ffb640 1 -- 172.21.15.155:0/1743455497 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+753 (secure 0 0 0) 0x7f663c003c50 con 0x7f664c100510 2024-09-19T06:11:05.676 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:11:05.676 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":1,"fsid":"9f2876d0-764d-11ef-bceb-c7b262605968","modified":"2024-09-19T06:09:23.791881Z","created":"2024-09-19T06:09:23.791881Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi053","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T06:11:05.676 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 1 2024-09-19T06:11:05.677 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.675+0000 7f6652551640 1 -- 172.21.15.155:0/1743455497 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f66380392d0 msgr2=0x7f663803b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:05.677 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.675+0000 7f6652551640 1 --2- 172.21.15.155:0/1743455497 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f66380392d0 0x7f663803b790 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f66340098c0 tx=0x7f6634006db0 comp rx=0 tx=0).stop 2024-09-19T06:11:05.677 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.675+0000 7f6652551640 1 -- 172.21.15.155:0/1743455497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f664c100510 msgr2=0x7f664c195bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:05.678 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.675+0000 7f6652551640 1 --2- 172.21.15.155:0/1743455497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f664c100510 0x7f664c195bd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f663c0046e0 tx=0x7f663c004710 comp rx=0 tx=0).stop 2024-09-19T06:11:05.678 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.676+0000 7f6652551640 1 -- 172.21.15.155:0/1743455497 shutdown_connections 2024-09-19T06:11:05.678 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.676+0000 7f6652551640 1 --2- 172.21.15.155:0/1743455497 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f66380392d0 0x7f663803b790 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:05.678 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.676+0000 7f6652551640 1 --2- 172.21.15.155:0/1743455497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f664c100510 0x7f664c195bd0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:05.678 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.676+0000 7f6652551640 1 -- 172.21.15.155:0/1743455497 >> 172.21.15.155:0/1743455497 conn(0x7f664c0fbc80 msgr2=0x7f664c0fc750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:05.678 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.676+0000 7f6652551640 1 -- 172.21.15.155:0/1743455497 shutdown_connections 2024-09-19T06:11:05.678 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:05.676+0000 7f6652551640 1 -- 172.21.15.155:0/1743455497 wait complete. 2024-09-19T06:11:05.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:05 smithi053 ceph-mon[27968]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:06.818 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:06 smithi053 ceph-mon[27968]: from='client.? 172.21.15.155:0/1743455497' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:07.214 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-19T06:11:07.214 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T06:11:07.456 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:11:07.501 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:11:07.641 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:07 smithi053 ceph-mon[27968]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:07.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:07.882+0000 7f9fb2f95640 1 -- 172.21.15.155:0/3140365170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fac100510 msgr2=0x7f9fac100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:07.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:07.882+0000 7f9fb2f95640 1 --2- 172.21.15.155:0/3140365170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fac100510 0x7f9fac100910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0009880 tx=0x7f9fa002a5d0 comp rx=0 tx=0).stop 2024-09-19T06:11:07.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:07.882+0000 7f9fb2f95640 1 -- 172.21.15.155:0/3140365170 shutdown_connections 2024-09-19T06:11:07.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:07.882+0000 7f9fb2f95640 1 --2- 172.21.15.155:0/3140365170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fac100510 0x7f9fac100910 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:07.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:07.882+0000 7f9fb2f95640 1 -- 172.21.15.155:0/3140365170 >> 172.21.15.155:0/3140365170 conn(0x7f9fac0fbc80 msgr2=0x7f9fac0fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:07.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:07.882+0000 7f9fb2f95640 1 -- 172.21.15.155:0/3140365170 shutdown_connections 2024-09-19T06:11:07.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:07.882+0000 7f9fb2f95640 1 -- 172.21.15.155:0/3140365170 wait complete. 2024-09-19T06:11:07.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:07.883+0000 7f9fb2f95640 1 Processor -- start 2024-09-19T06:11:07.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:07.883+0000 7f9fb2f95640 1 -- start start 2024-09-19T06:11:07.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:07.883+0000 7f9fb2f95640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fac100510 0x7f9fac06a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:07.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:07.883+0000 7f9fb2f95640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac1064d0 con 0x7f9fac100510 2024-09-19T06:11:07.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:07.883+0000 7f9fb0d31640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fac100510 0x7f9fac06a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:07.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:07.883+0000 7f9fb0d31640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fac100510 0x7f9fac06a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.155:35438/0 (socket says 172.21.15.155:35438) 2024-09-19T06:11:07.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:07.883+0000 7f9fb0d31640 1 -- 172.21.15.155:0/485025382 learned_addr learned my addr 172.21.15.155:0/485025382 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-19T06:11:07.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:07.884+0000 7f9fb0d31640 1 -- 172.21.15.155:0/485025382 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fac1999f0 con 0x7f9fac100510 2024-09-19T06:11:07.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:07.884+0000 7f9fb0d31640 1 --2- 172.21.15.155:0/485025382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fac100510 0x7f9fac06a990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00046e0 tx=0x7f9fa0004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:07.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:07.884+0000 7f9fa9ffb640 1 -- 172.21.15.155:0/485025382 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f9fa003f070 con 0x7f9fac100510 2024-09-19T06:11:07.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:07.884+0000 7f9fa9ffb640 1 -- 172.21.15.155:0/485025382 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fa0003aa0 con 0x7f9fac100510 2024-09-19T06:11:07.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:07.884+0000 7f9fb2f95640 1 -- 172.21.15.155:0/485025382 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fac199bc0 con 0x7f9fac100510 2024-09-19T06:11:07.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:07.884+0000 7f9fa9ffb640 1 -- 172.21.15.155:0/485025382 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f9fa0035030 con 0x7f9fac100510 2024-09-19T06:11:07.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:07.885+0000 7f9fb2f95640 1 -- 172.21.15.155:0/485025382 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fac199fe0 con 0x7f9fac100510 2024-09-19T06:11:07.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:07.886+0000 7f9fa9ffb640 1 -- 172.21.15.155:0/485025382 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46532+0+0 (secure 0 0 0) 0x7f9fa0042070 con 0x7f9fac100510 2024-09-19T06:11:07.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:07.886+0000 7f9fb2f95640 1 -- 172.21.15.155:0/485025382 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fac106c50 con 0x7f9fac100510 2024-09-19T06:11:07.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:07.886+0000 7f9fa9ffb640 1 --2- 172.21.15.155:0/485025382 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f9f9c0392d0 0x7f9f9c03b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:07.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:07.886+0000 7f9fa9ffb640 1 -- 172.21.15.155:0/485025382 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9fa006d280 con 0x7f9fac100510 2024-09-19T06:11:07.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:07.886+0000 7f9fabfff640 1 --2- 172.21.15.155:0/485025382 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f9f9c0392d0 0x7f9f9c03b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:07.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:07.887+0000 7f9fabfff640 1 --2- 172.21.15.155:0/485025382 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f9f9c0392d0 0x7f9f9c03b790 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9f980098c0 tx=0x7f9f98006db0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:07.893 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:07.892+0000 7f9fa9ffb640 1 -- 172.21.15.155:0/485025382 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9fa0042360 con 0x7f9fac100510 2024-09-19T06:11:08.125 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:08.123+0000 7f9fb2f95640 1 -- 172.21.15.155:0/485025382 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9fac02d950 con 0x7f9fac100510 2024-09-19T06:11:08.125 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:08.124+0000 7f9fa9ffb640 1 -- 172.21.15.155:0/485025382 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+753 (secure 0 0 0) 0x7f9fa002f300 con 0x7f9fac100510 2024-09-19T06:11:08.126 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:11:08.126 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":1,"fsid":"9f2876d0-764d-11ef-bceb-c7b262605968","modified":"2024-09-19T06:09:23.791881Z","created":"2024-09-19T06:09:23.791881Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi053","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T06:11:08.126 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 1 2024-09-19T06:11:08.128 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:08.126+0000 7f9fb2f95640 1 -- 172.21.15.155:0/485025382 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f9f9c0392d0 msgr2=0x7f9f9c03b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:08.128 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:08.126+0000 7f9fb2f95640 1 --2- 172.21.15.155:0/485025382 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f9f9c0392d0 0x7f9f9c03b790 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9f980098c0 tx=0x7f9f98006db0 comp rx=0 tx=0).stop 2024-09-19T06:11:08.128 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:08.126+0000 7f9fb2f95640 1 -- 172.21.15.155:0/485025382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fac100510 msgr2=0x7f9fac06a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:08.128 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:08.126+0000 7f9fb2f95640 1 --2- 172.21.15.155:0/485025382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fac100510 0x7f9fac06a990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00046e0 tx=0x7f9fa0004710 comp rx=0 tx=0).stop 2024-09-19T06:11:08.128 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:08.126+0000 7f9fb2f95640 1 -- 172.21.15.155:0/485025382 shutdown_connections 2024-09-19T06:11:08.128 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:08.126+0000 7f9fb2f95640 1 --2- 172.21.15.155:0/485025382 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f9f9c0392d0 0x7f9f9c03b790 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:08.128 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:08.126+0000 7f9fb2f95640 1 --2- 172.21.15.155:0/485025382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fac100510 0x7f9fac06a990 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:08.129 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:08.126+0000 7f9fb2f95640 1 -- 172.21.15.155:0/485025382 >> 172.21.15.155:0/485025382 conn(0x7f9fac0fbc80 msgr2=0x7f9fac0fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:08.129 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:08.126+0000 7f9fb2f95640 1 -- 172.21.15.155:0/485025382 shutdown_connections 2024-09-19T06:11:08.129 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:08.126+0000 7f9fb2f95640 1 -- 172.21.15.155:0/485025382 wait complete. 2024-09-19T06:11:08.616 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:08 smithi053 ceph-mon[27968]: from='client.? 172.21.15.155:0/485025382' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:09.799 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-19T06:11:09.799 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T06:11:09.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:09 smithi053 ceph-mon[27968]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:09.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:09 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:09.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:09 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:09.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:09 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:09.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:09 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:09.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:09 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:09.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:09 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:09.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:09 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:09.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:09 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:09.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:09 smithi053 ceph-mon[27968]: Deploying daemon prometheus.smithi053 on smithi053 2024-09-19T06:11:10.035 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:11:10.079 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:11:10.488 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.481+0000 7f3a627d9640 1 -- 172.21.15.155:0/592644174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a5c100510 msgr2=0x7f3a5c100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:10.488 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.481+0000 7f3a627d9640 1 --2- 172.21.15.155:0/592644174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a5c100510 0x7f3a5c100910 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c009880 tx=0x7f3a4c02a5d0 comp rx=0 tx=0).stop 2024-09-19T06:11:10.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.481+0000 7f3a627d9640 1 -- 172.21.15.155:0/592644174 shutdown_connections 2024-09-19T06:11:10.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.481+0000 7f3a627d9640 1 --2- 172.21.15.155:0/592644174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a5c100510 0x7f3a5c100910 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:10.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.481+0000 7f3a627d9640 1 -- 172.21.15.155:0/592644174 >> 172.21.15.155:0/592644174 conn(0x7f3a5c0fbc80 msgr2=0x7f3a5c0fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:10.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.481+0000 7f3a627d9640 1 -- 172.21.15.155:0/592644174 shutdown_connections 2024-09-19T06:11:10.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.481+0000 7f3a627d9640 1 -- 172.21.15.155:0/592644174 wait complete. 2024-09-19T06:11:10.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.482+0000 7f3a627d9640 1 Processor -- start 2024-09-19T06:11:10.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.482+0000 7f3a627d9640 1 -- start start 2024-09-19T06:11:10.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.482+0000 7f3a627d9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a5c100510 0x7f3a5c0fdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:10.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.482+0000 7f3a627d9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5c1064d0 con 0x7f3a5c100510 2024-09-19T06:11:10.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.482+0000 7f3a5bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a5c100510 0x7f3a5c0fdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:10.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.482+0000 7f3a5bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a5c100510 0x7f3a5c0fdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.155:38674/0 (socket says 172.21.15.155:38674) 2024-09-19T06:11:10.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.482+0000 7f3a5bfff640 1 -- 172.21.15.155:0/2763111410 learned_addr learned my addr 172.21.15.155:0/2763111410 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-19T06:11:10.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.483+0000 7f3a5bfff640 1 -- 172.21.15.155:0/2763111410 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5c0fe0a0 con 0x7f3a5c100510 2024-09-19T06:11:10.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.483+0000 7f3a5bfff640 1 --2- 172.21.15.155:0/2763111410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a5c100510 0x7f3a5c0fdb60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c0046e0 tx=0x7f3a4c004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:10.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.483+0000 7f3a597fa640 1 -- 172.21.15.155:0/2763111410 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f3a4c03f070 con 0x7f3a5c100510 2024-09-19T06:11:10.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.483+0000 7f3a627d9640 1 -- 172.21.15.155:0/2763111410 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a5c0fc290 con 0x7f3a5c100510 2024-09-19T06:11:10.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.483+0000 7f3a597fa640 1 -- 172.21.15.155:0/2763111410 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a4c003aa0 con 0x7f3a5c100510 2024-09-19T06:11:10.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.483+0000 7f3a597fa640 1 -- 172.21.15.155:0/2763111410 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f3a4c035030 con 0x7f3a5c100510 2024-09-19T06:11:10.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.483+0000 7f3a627d9640 1 -- 172.21.15.155:0/2763111410 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a5c0fc7d0 con 0x7f3a5c100510 2024-09-19T06:11:10.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.484+0000 7f3a627d9640 1 -- 172.21.15.155:0/2763111410 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a5c106c50 con 0x7f3a5c100510 2024-09-19T06:11:10.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.484+0000 7f3a597fa640 1 -- 172.21.15.155:0/2763111410 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46532+0+0 (secure 0 0 0) 0x7f3a4c042070 con 0x7f3a5c100510 2024-09-19T06:11:10.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.484+0000 7f3a597fa640 1 --2- 172.21.15.155:0/2763111410 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f3a440392d0 0x7f3a4403b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:10.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.484+0000 7f3a597fa640 1 -- 172.21.15.155:0/2763111410 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3a4c06d220 con 0x7f3a5c100510 2024-09-19T06:11:10.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.484+0000 7f3a5b7fe640 1 --2- 172.21.15.155:0/2763111410 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f3a440392d0 0x7f3a4403b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:10.491 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.485+0000 7f3a5b7fe640 1 --2- 172.21.15.155:0/2763111410 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f3a440392d0 0x7f3a4403b790 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3a500098c0 tx=0x7f3a50006db0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:10.491 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.489+0000 7f3a597fa640 1 -- 172.21.15.155:0/2763111410 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3a4c040260 con 0x7f3a5c100510 2024-09-19T06:11:10.723 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.721+0000 7f3a627d9640 1 -- 172.21.15.155:0/2763111410 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f3a5c1901c0 con 0x7f3a5c100510 2024-09-19T06:11:10.723 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.722+0000 7f3a597fa640 1 -- 172.21.15.155:0/2763111410 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+753 (secure 0 0 0) 0x7f3a4c039350 con 0x7f3a5c100510 2024-09-19T06:11:10.724 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:11:10.724 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":1,"fsid":"9f2876d0-764d-11ef-bceb-c7b262605968","modified":"2024-09-19T06:09:23.791881Z","created":"2024-09-19T06:09:23.791881Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi053","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T06:11:10.724 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 1 2024-09-19T06:11:10.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.724+0000 7f3a627d9640 1 -- 172.21.15.155:0/2763111410 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f3a440392d0 msgr2=0x7f3a4403b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:10.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.724+0000 7f3a627d9640 1 --2- 172.21.15.155:0/2763111410 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f3a440392d0 0x7f3a4403b790 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3a500098c0 tx=0x7f3a50006db0 comp rx=0 tx=0).stop 2024-09-19T06:11:10.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.724+0000 7f3a627d9640 1 -- 172.21.15.155:0/2763111410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a5c100510 msgr2=0x7f3a5c0fdb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:10.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.724+0000 7f3a627d9640 1 --2- 172.21.15.155:0/2763111410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a5c100510 0x7f3a5c0fdb60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c0046e0 tx=0x7f3a4c004710 comp rx=0 tx=0).stop 2024-09-19T06:11:10.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.724+0000 7f3a627d9640 1 -- 172.21.15.155:0/2763111410 shutdown_connections 2024-09-19T06:11:10.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.724+0000 7f3a627d9640 1 --2- 172.21.15.155:0/2763111410 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f3a440392d0 0x7f3a4403b790 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:10.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.724+0000 7f3a627d9640 1 --2- 172.21.15.155:0/2763111410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a5c100510 0x7f3a5c0fdb60 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:10.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.724+0000 7f3a627d9640 1 -- 172.21.15.155:0/2763111410 >> 172.21.15.155:0/2763111410 conn(0x7f3a5c0fbc80 msgr2=0x7f3a5c06a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:10.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.724+0000 7f3a627d9640 1 -- 172.21.15.155:0/2763111410 shutdown_connections 2024-09-19T06:11:10.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:10.724+0000 7f3a627d9640 1 -- 172.21.15.155:0/2763111410 wait complete. 2024-09-19T06:11:11.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:11 smithi053 ceph-mon[27968]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:11.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:11 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:11.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:11 smithi053 ceph-mon[27968]: from='client.? 172.21.15.155:0/2763111410' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:12.295 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-19T06:11:12.295 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T06:11:12.540 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:11:12.585 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:11:13.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:12.987+0000 7f5955833640 1 -- 172.21.15.155:0/3767489906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5950102130 msgr2=0x7f5950104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:13.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:12.987+0000 7f5955833640 1 --2- 172.21.15.155:0/3767489906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5950102130 0x7f5950104520 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5940009c20 tx=0x7f594002aa90 comp rx=0 tx=0).stop 2024-09-19T06:11:13.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:12.987+0000 7f5955833640 1 -- 172.21.15.155:0/3767489906 shutdown_connections 2024-09-19T06:11:13.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:12.987+0000 7f5955833640 1 --2- 172.21.15.155:0/3767489906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5950102130 0x7f5950104520 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:12.987+0000 7f5955833640 1 -- 172.21.15.155:0/3767489906 >> 172.21.15.155:0/3767489906 conn(0x7f59500fbcd0 msgr2=0x7f59500fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:13.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:12.987+0000 7f5955833640 1 -- 172.21.15.155:0/3767489906 shutdown_connections 2024-09-19T06:11:13.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:12.987+0000 7f5955833640 1 -- 172.21.15.155:0/3767489906 wait complete. 2024-09-19T06:11:13.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:12.987+0000 7f5955833640 1 Processor -- start 2024-09-19T06:11:13.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:12.988+0000 7f5955833640 1 -- start start 2024-09-19T06:11:13.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:12.988+0000 7f5955833640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5950102130 0x7f5950196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:13.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:12.988+0000 7f5955833640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5950107590 con 0x7f5950102130 2024-09-19T06:11:13.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:12.988+0000 7f594effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5950102130 0x7f5950196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:13.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:12.988+0000 7f594effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5950102130 0x7f5950196c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.155:38684/0 (socket says 172.21.15.155:38684) 2024-09-19T06:11:13.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:12.988+0000 7f594effd640 1 -- 172.21.15.155:0/3123150393 learned_addr learned my addr 172.21.15.155:0/3123150393 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-19T06:11:13.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:12.988+0000 7f594effd640 1 -- 172.21.15.155:0/3123150393 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59501971d0 con 0x7f5950102130 2024-09-19T06:11:13.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:12.989+0000 7f594effd640 1 --2- 172.21.15.155:0/3123150393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5950102130 0x7f5950196c90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5940006010 tx=0x7f594002f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:13.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:12.989+0000 7f5954831640 1 -- 172.21.15.155:0/3123150393 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f5940047070 con 0x7f5950102130 2024-09-19T06:11:13.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:12.989+0000 7f5954831640 1 -- 172.21.15.155:0/3123150393 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5940003e20 con 0x7f5950102130 2024-09-19T06:11:13.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:12.989+0000 7f5955833640 1 -- 172.21.15.155:0/3123150393 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5950197460 con 0x7f5950102130 2024-09-19T06:11:13.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:12.989+0000 7f5954831640 1 -- 172.21.15.155:0/3123150393 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f5940033050 con 0x7f5950102130 2024-09-19T06:11:13.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:12.989+0000 7f5955833640 1 -- 172.21.15.155:0/3123150393 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f595019a150 con 0x7f5950102130 2024-09-19T06:11:13.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:12.990+0000 7f5954831640 1 -- 172.21.15.155:0/3123150393 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46532+0+0 (secure 0 0 0) 0x7f594004a070 con 0x7f5950102130 2024-09-19T06:11:13.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:12.990+0000 7f5955833640 1 -- 172.21.15.155:0/3123150393 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5950107d10 con 0x7f5950102130 2024-09-19T06:11:13.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:12.990+0000 7f5954831640 1 --2- 172.21.15.155:0/3123150393 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f59380392d0 0x7f593803b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:13.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:12.990+0000 7f5954831640 1 -- 172.21.15.155:0/3123150393 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f594003e030 con 0x7f5950102130 2024-09-19T06:11:13.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:12.990+0000 7f594e7fc640 1 --2- 172.21.15.155:0/3123150393 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f59380392d0 0x7f593803b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:13.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:12.991+0000 7f594e7fc640 1 --2- 172.21.15.155:0/3123150393 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f59380392d0 0x7f593803b790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f59440098c0 tx=0x7f5944006db0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:13.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:12.995+0000 7f5954831640 1 -- 172.21.15.155:0/3123150393 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f59400098b0 con 0x7f5950102130 2024-09-19T06:11:13.227 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:13.226+0000 7f5955833640 1 -- 172.21.15.155:0/3123150393 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f595002d7c0 con 0x7f5950102130 2024-09-19T06:11:13.228 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:13.227+0000 7f5954831640 1 -- 172.21.15.155:0/3123150393 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+753 (secure 0 0 0) 0x7f594004b3b0 con 0x7f5950102130 2024-09-19T06:11:13.228 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:11:13.228 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":1,"fsid":"9f2876d0-764d-11ef-bceb-c7b262605968","modified":"2024-09-19T06:09:23.791881Z","created":"2024-09-19T06:09:23.791881Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi053","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T06:11:13.229 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 1 2024-09-19T06:11:13.230 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:13.228+0000 7f5955833640 1 -- 172.21.15.155:0/3123150393 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f59380392d0 msgr2=0x7f593803b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:13.230 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:13.228+0000 7f5955833640 1 --2- 172.21.15.155:0/3123150393 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f59380392d0 0x7f593803b790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f59440098c0 tx=0x7f5944006db0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.230 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:13.228+0000 7f5955833640 1 -- 172.21.15.155:0/3123150393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5950102130 msgr2=0x7f5950196c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:13.230 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:13.228+0000 7f5955833640 1 --2- 172.21.15.155:0/3123150393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5950102130 0x7f5950196c90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5940006010 tx=0x7f594002f8a0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.230 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:13.229+0000 7f5955833640 1 -- 172.21.15.155:0/3123150393 shutdown_connections 2024-09-19T06:11:13.230 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:13.229+0000 7f5955833640 1 --2- 172.21.15.155:0/3123150393 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f59380392d0 0x7f593803b790 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.230 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:13.229+0000 7f5955833640 1 --2- 172.21.15.155:0/3123150393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5950102130 0x7f5950196c90 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.231 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:13.229+0000 7f5955833640 1 -- 172.21.15.155:0/3123150393 >> 172.21.15.155:0/3123150393 conn(0x7f59500fbcd0 msgr2=0x7f59500fc780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:13.231 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:13.229+0000 7f5955833640 1 -- 172.21.15.155:0/3123150393 shutdown_connections 2024-09-19T06:11:13.231 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:13.229+0000 7f5955833640 1 -- 172.21.15.155:0/3123150393 wait complete. 2024-09-19T06:11:13.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:13 smithi053 ceph-mon[27968]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:13.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:13 smithi053 ceph-mon[27968]: from='client.? 172.21.15.155:0/3123150393' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:14.791 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-19T06:11:14.791 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T06:11:15.035 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:11:15.080 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:11:15.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.466+0000 7f5ce6b52640 1 -- 172.21.15.155:0/3038060737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ce0101a80 msgr2=0x7f5ce0101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:15.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.466+0000 7f5ce6b52640 1 --2- 172.21.15.155:0/3038060737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ce0101a80 0x7f5ce0101e80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4009880 tx=0x7f5cd402a5d0 comp rx=0 tx=0).stop 2024-09-19T06:11:15.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.466+0000 7f5ce6b52640 1 -- 172.21.15.155:0/3038060737 shutdown_connections 2024-09-19T06:11:15.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.466+0000 7f5ce6b52640 1 --2- 172.21.15.155:0/3038060737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ce0101a80 0x7f5ce0101e80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:15.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.466+0000 7f5ce6b52640 1 -- 172.21.15.155:0/3038060737 >> 172.21.15.155:0/3038060737 conn(0x7f5ce00fb400 msgr2=0x7f5ce00fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:15.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.466+0000 7f5ce6b52640 1 -- 172.21.15.155:0/3038060737 shutdown_connections 2024-09-19T06:11:15.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.466+0000 7f5ce6b52640 1 -- 172.21.15.155:0/3038060737 wait complete. 2024-09-19T06:11:15.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.467+0000 7f5ce6b52640 1 Processor -- start 2024-09-19T06:11:15.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.467+0000 7f5ce6b52640 1 -- start start 2024-09-19T06:11:15.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.467+0000 7f5ce6b52640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ce0101a80 0x7f5ce00fd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:15.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.467+0000 7f5ce6b52640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce01064c0 con 0x7f5ce0101a80 2024-09-19T06:11:15.470 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.467+0000 7f5ce48ee640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ce0101a80 0x7f5ce00fd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:15.470 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.467+0000 7f5ce48ee640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ce0101a80 0x7f5ce00fd2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.155:38704/0 (socket says 172.21.15.155:38704) 2024-09-19T06:11:15.470 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.467+0000 7f5ce48ee640 1 -- 172.21.15.155:0/3736037119 learned_addr learned my addr 172.21.15.155:0/3736037119 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-19T06:11:15.470 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.468+0000 7f5ce48ee640 1 -- 172.21.15.155:0/3736037119 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce00fd800 con 0x7f5ce0101a80 2024-09-19T06:11:15.470 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.468+0000 7f5ce48ee640 1 --2- 172.21.15.155:0/3736037119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ce0101a80 0x7f5ce00fd2c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40046e0 tx=0x7f5cd4004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:15.471 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.468+0000 7f5cddffb640 1 -- 172.21.15.155:0/3736037119 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f5cd403f070 con 0x7f5ce0101a80 2024-09-19T06:11:15.471 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.468+0000 7f5cddffb640 1 -- 172.21.15.155:0/3736037119 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5cd4003aa0 con 0x7f5ce0101a80 2024-09-19T06:11:15.471 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.468+0000 7f5ce6b52640 1 -- 172.21.15.155:0/3736037119 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ce00fb9f0 con 0x7f5ce0101a80 2024-09-19T06:11:15.471 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.468+0000 7f5cddffb640 1 -- 172.21.15.155:0/3736037119 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f5cd4035030 con 0x7f5ce0101a80 2024-09-19T06:11:15.471 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.468+0000 7f5ce6b52640 1 -- 172.21.15.155:0/3736037119 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ce00fbf30 con 0x7f5ce0101a80 2024-09-19T06:11:15.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.469+0000 7f5cddffb640 1 -- 172.21.15.155:0/3736037119 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46532+0+0 (secure 0 0 0) 0x7f5cd4042070 con 0x7f5ce0101a80 2024-09-19T06:11:15.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.469+0000 7f5ce6b52640 1 -- 172.21.15.155:0/3736037119 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ce0106c40 con 0x7f5ce0101a80 2024-09-19T06:11:15.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.469+0000 7f5cddffb640 1 --2- 172.21.15.155:0/3736037119 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f5cd00392d0 0x7f5cd003b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:15.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.469+0000 7f5cddffb640 1 -- 172.21.15.155:0/3736037119 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5cd406d210 con 0x7f5ce0101a80 2024-09-19T06:11:15.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.470+0000 7f5cdffff640 1 --2- 172.21.15.155:0/3736037119 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f5cd00392d0 0x7f5cd003b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:15.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.470+0000 7f5cdffff640 1 --2- 172.21.15.155:0/3736037119 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f5cd00392d0 0x7f5cd003b790 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc0098c0 tx=0x7f5ccc006db0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:15.476 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.475+0000 7f5cddffb640 1 -- 172.21.15.155:0/3736037119 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5cd4040260 con 0x7f5ce0101a80 2024-09-19T06:11:15.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:15 smithi053 ceph-mon[27968]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:15.703 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.701+0000 7f5ce6b52640 1 -- 172.21.15.155:0/3736037119 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f5ce01901a0 con 0x7f5ce0101a80 2024-09-19T06:11:15.703 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.702+0000 7f5cddffb640 1 -- 172.21.15.155:0/3736037119 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+753 (secure 0 0 0) 0x7f5cd4039350 con 0x7f5ce0101a80 2024-09-19T06:11:15.704 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:11:15.704 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":1,"fsid":"9f2876d0-764d-11ef-bceb-c7b262605968","modified":"2024-09-19T06:09:23.791881Z","created":"2024-09-19T06:09:23.791881Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi053","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T06:11:15.704 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 1 2024-09-19T06:11:15.706 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.704+0000 7f5ce6b52640 1 -- 172.21.15.155:0/3736037119 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f5cd00392d0 msgr2=0x7f5cd003b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:15.706 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.704+0000 7f5ce6b52640 1 --2- 172.21.15.155:0/3736037119 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f5cd00392d0 0x7f5cd003b790 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc0098c0 tx=0x7f5ccc006db0 comp rx=0 tx=0).stop 2024-09-19T06:11:15.706 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.704+0000 7f5ce6b52640 1 -- 172.21.15.155:0/3736037119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ce0101a80 msgr2=0x7f5ce00fd2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:15.706 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.704+0000 7f5ce6b52640 1 --2- 172.21.15.155:0/3736037119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ce0101a80 0x7f5ce00fd2c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40046e0 tx=0x7f5cd4004710 comp rx=0 tx=0).stop 2024-09-19T06:11:15.706 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.704+0000 7f5ce6b52640 1 -- 172.21.15.155:0/3736037119 shutdown_connections 2024-09-19T06:11:15.706 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.704+0000 7f5ce6b52640 1 --2- 172.21.15.155:0/3736037119 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f5cd00392d0 0x7f5cd003b790 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:15.706 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.704+0000 7f5ce6b52640 1 --2- 172.21.15.155:0/3736037119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ce0101a80 0x7f5ce00fd2c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:15.706 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.704+0000 7f5ce6b52640 1 -- 172.21.15.155:0/3736037119 >> 172.21.15.155:0/3736037119 conn(0x7f5ce00fb400 msgr2=0x7f5ce00ffb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:15.706 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.704+0000 7f5ce6b52640 1 -- 172.21.15.155:0/3736037119 shutdown_connections 2024-09-19T06:11:15.706 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:15.704+0000 7f5ce6b52640 1 -- 172.21.15.155:0/3736037119 wait complete. 2024-09-19T06:11:16.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:16 smithi053 ceph-mon[27968]: from='client.? 172.21.15.155:0/3736037119' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:17.338 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-19T06:11:17.339 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T06:11:17.581 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:11:17.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:17 smithi053 ceph-mon[27968]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:17.625 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:11:18.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.006+0000 7f4009634640 1 -- 172.21.15.155:0/1137521668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4004100540 msgr2=0x7f4004100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:18.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.006+0000 7f4009634640 1 --2- 172.21.15.155:0/1137521668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4004100540 0x7f4004100940 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8009880 tx=0x7f3ff802a5d0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.006+0000 7f4009634640 1 -- 172.21.15.155:0/1137521668 shutdown_connections 2024-09-19T06:11:18.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.006+0000 7f4009634640 1 --2- 172.21.15.155:0/1137521668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4004100540 0x7f4004100940 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.006+0000 7f4009634640 1 -- 172.21.15.155:0/1137521668 >> 172.21.15.155:0/1137521668 conn(0x7f40040fbcd0 msgr2=0x7f40040fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:18.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.006+0000 7f4009634640 1 -- 172.21.15.155:0/1137521668 shutdown_connections 2024-09-19T06:11:18.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.006+0000 7f4009634640 1 -- 172.21.15.155:0/1137521668 wait complete. 2024-09-19T06:11:18.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.007+0000 7f4009634640 1 Processor -- start 2024-09-19T06:11:18.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.008+0000 7f4009634640 1 -- start start 2024-09-19T06:11:18.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.008+0000 7f4009634640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4004100540 0x7f4004196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:18.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.008+0000 7f4009634640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f40041064a0 con 0x7f4004100540 2024-09-19T06:11:18.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.008+0000 7f4002ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4004100540 0x7f4004196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:18.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.008+0000 7f4002ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4004100540 0x7f4004196cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.155:38726/0 (socket says 172.21.15.155:38726) 2024-09-19T06:11:18.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.008+0000 7f4002ffd640 1 -- 172.21.15.155:0/4169899952 learned_addr learned my addr 172.21.15.155:0/4169899952 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-19T06:11:18.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.008+0000 7f4002ffd640 1 -- 172.21.15.155:0/4169899952 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4004197200 con 0x7f4004100540 2024-09-19T06:11:18.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.009+0000 7f4002ffd640 1 --2- 172.21.15.155:0/4169899952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4004100540 0x7f4004196cc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3ff80046e0 tx=0x7f3ff8004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:18.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.009+0000 7f3febfff640 1 -- 172.21.15.155:0/4169899952 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f3ff803f070 con 0x7f4004100540 2024-09-19T06:11:18.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.009+0000 7f4009634640 1 -- 172.21.15.155:0/4169899952 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4004197490 con 0x7f4004100540 2024-09-19T06:11:18.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.009+0000 7f3febfff640 1 -- 172.21.15.155:0/4169899952 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ff8003aa0 con 0x7f4004100540 2024-09-19T06:11:18.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.009+0000 7f3febfff640 1 -- 172.21.15.155:0/4169899952 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f3ff8035030 con 0x7f4004100540 2024-09-19T06:11:18.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.009+0000 7f4009634640 1 -- 172.21.15.155:0/4169899952 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f400419a180 con 0x7f4004100540 2024-09-19T06:11:18.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.010+0000 7f4009634640 1 -- 172.21.15.155:0/4169899952 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4004106c20 con 0x7f4004100540 2024-09-19T06:11:18.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.010+0000 7f3febfff640 1 -- 172.21.15.155:0/4169899952 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46532+0+0 (secure 0 0 0) 0x7f3ff8042070 con 0x7f4004100540 2024-09-19T06:11:18.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.010+0000 7f3febfff640 1 --2- 172.21.15.155:0/4169899952 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f3fec0392d0 0x7f3fec03b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:18.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.010+0000 7f3febfff640 1 -- 172.21.15.155:0/4169899952 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3ff806d220 con 0x7f4004100540 2024-09-19T06:11:18.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.010+0000 7f40027fc640 1 --2- 172.21.15.155:0/4169899952 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f3fec0392d0 0x7f3fec03b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:18.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.011+0000 7f40027fc640 1 --2- 172.21.15.155:0/4169899952 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f3fec0392d0 0x7f3fec03b790 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400ad30 tx=0x7f3ff40093f0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:18.017 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.015+0000 7f3febfff640 1 -- 172.21.15.155:0/4169899952 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3ff8042360 con 0x7f4004100540 2024-09-19T06:11:18.245 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.243+0000 7f4009634640 1 -- 172.21.15.155:0/4169899952 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f400402d7c0 con 0x7f4004100540 2024-09-19T06:11:18.245 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.244+0000 7f3febfff640 1 -- 172.21.15.155:0/4169899952 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+753 (secure 0 0 0) 0x7f3ff8040260 con 0x7f4004100540 2024-09-19T06:11:18.246 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:11:18.246 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":1,"fsid":"9f2876d0-764d-11ef-bceb-c7b262605968","modified":"2024-09-19T06:09:23.791881Z","created":"2024-09-19T06:09:23.791881Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi053","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T06:11:18.246 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 1 2024-09-19T06:11:18.247 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.246+0000 7f4009634640 1 -- 172.21.15.155:0/4169899952 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f3fec0392d0 msgr2=0x7f3fec03b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:18.248 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.246+0000 7f4009634640 1 --2- 172.21.15.155:0/4169899952 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f3fec0392d0 0x7f3fec03b790 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400ad30 tx=0x7f3ff40093f0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.248 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.246+0000 7f4009634640 1 -- 172.21.15.155:0/4169899952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4004100540 msgr2=0x7f4004196cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:18.248 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.246+0000 7f4009634640 1 --2- 172.21.15.155:0/4169899952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4004100540 0x7f4004196cc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3ff80046e0 tx=0x7f3ff8004710 comp rx=0 tx=0).stop 2024-09-19T06:11:18.248 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.246+0000 7f4009634640 1 -- 172.21.15.155:0/4169899952 shutdown_connections 2024-09-19T06:11:18.248 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.246+0000 7f4009634640 1 --2- 172.21.15.155:0/4169899952 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f3fec0392d0 0x7f3fec03b790 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.248 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.246+0000 7f4009634640 1 --2- 172.21.15.155:0/4169899952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4004100540 0x7f4004196cc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.248 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.246+0000 7f4009634640 1 -- 172.21.15.155:0/4169899952 >> 172.21.15.155:0/4169899952 conn(0x7f40040fbcd0 msgr2=0x7f4004069b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:18.248 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.246+0000 7f4009634640 1 -- 172.21.15.155:0/4169899952 shutdown_connections 2024-09-19T06:11:18.248 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:18.246+0000 7f4009634640 1 -- 172.21.15.155:0/4169899952 wait complete. 2024-09-19T06:11:18.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:18 smithi053 ceph-mon[27968]: from='client.? 172.21.15.155:0/4169899952' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:19.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:19 smithi053 ceph-mon[27968]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:19.781 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-19T06:11:19.781 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T06:11:20.044 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:11:20.088 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:11:20.502 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.497+0000 7f271c767640 1 -- 172.21.15.155:0/3948889707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2714100540 msgr2=0x7f2714100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:20.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.497+0000 7f271c767640 1 --2- 172.21.15.155:0/3948889707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2714100540 0x7f2714100940 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2710009880 tx=0x7f271002a5d0 comp rx=0 tx=0).stop 2024-09-19T06:11:20.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.497+0000 7f271c767640 1 -- 172.21.15.155:0/3948889707 shutdown_connections 2024-09-19T06:11:20.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.497+0000 7f271c767640 1 --2- 172.21.15.155:0/3948889707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2714100540 0x7f2714100940 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:20.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.497+0000 7f271c767640 1 -- 172.21.15.155:0/3948889707 >> 172.21.15.155:0/3948889707 conn(0x7f27140fbcd0 msgr2=0x7f27140fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:20.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.497+0000 7f271c767640 1 -- 172.21.15.155:0/3948889707 shutdown_connections 2024-09-19T06:11:20.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.497+0000 7f271c767640 1 -- 172.21.15.155:0/3948889707 wait complete. 2024-09-19T06:11:20.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.498+0000 7f271c767640 1 Processor -- start 2024-09-19T06:11:20.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.498+0000 7f271c767640 1 -- start start 2024-09-19T06:11:20.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.498+0000 7f271c767640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2714100540 0x7f2714196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:20.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.498+0000 7f271c767640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f27141064a0 con 0x7f2714100540 2024-09-19T06:11:20.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.498+0000 7f271a503640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2714100540 0x7f2714196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:20.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.498+0000 7f271a503640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2714100540 0x7f2714196cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.155:35912/0 (socket says 172.21.15.155:35912) 2024-09-19T06:11:20.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.498+0000 7f271a503640 1 -- 172.21.15.155:0/2894730809 learned_addr learned my addr 172.21.15.155:0/2894730809 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-19T06:11:20.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.498+0000 7f271a503640 1 -- 172.21.15.155:0/2894730809 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2714197200 con 0x7f2714100540 2024-09-19T06:11:20.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.499+0000 7f271a503640 1 --2- 172.21.15.155:0/2894730809 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2714100540 0x7f2714196cc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f27100046e0 tx=0x7f2710004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:20.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.499+0000 7f27077fe640 1 -- 172.21.15.155:0/2894730809 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f271003f070 con 0x7f2714100540 2024-09-19T06:11:20.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.499+0000 7f271c767640 1 -- 172.21.15.155:0/2894730809 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2714197490 con 0x7f2714100540 2024-09-19T06:11:20.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.499+0000 7f27077fe640 1 -- 172.21.15.155:0/2894730809 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2710003aa0 con 0x7f2714100540 2024-09-19T06:11:20.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.499+0000 7f27077fe640 1 -- 172.21.15.155:0/2894730809 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f2710035030 con 0x7f2714100540 2024-09-19T06:11:20.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.499+0000 7f271c767640 1 -- 172.21.15.155:0/2894730809 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f271419a180 con 0x7f2714100540 2024-09-19T06:11:20.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.500+0000 7f271c767640 1 -- 172.21.15.155:0/2894730809 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2714106c20 con 0x7f2714100540 2024-09-19T06:11:20.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.500+0000 7f27077fe640 1 -- 172.21.15.155:0/2894730809 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46532+0+0 (secure 0 0 0) 0x7f2710042070 con 0x7f2714100540 2024-09-19T06:11:20.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.500+0000 7f27077fe640 1 --2- 172.21.15.155:0/2894730809 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f27000392d0 0x7f270003b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:20.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.500+0000 7f27077fe640 1 -- 172.21.15.155:0/2894730809 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f271006d1d0 con 0x7f2714100540 2024-09-19T06:11:20.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.500+0000 7f2719d02640 1 --2- 172.21.15.155:0/2894730809 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f27000392d0 0x7f270003b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:20.505 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.501+0000 7f2719d02640 1 --2- 172.21.15.155:0/2894730809 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f27000392d0 0x7f270003b790 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f27080098c0 tx=0x7f2708006db0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:20.508 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.505+0000 7f27077fe640 1 -- 172.21.15.155:0/2894730809 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2710040260 con 0x7f2714100540 2024-09-19T06:11:20.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.733+0000 7f271c767640 1 -- 172.21.15.155:0/2894730809 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f271402d7c0 con 0x7f2714100540 2024-09-19T06:11:20.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.733+0000 7f27077fe640 1 -- 172.21.15.155:0/2894730809 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+753 (secure 0 0 0) 0x7f2710039350 con 0x7f2714100540 2024-09-19T06:11:20.735 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:11:20.735 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":1,"fsid":"9f2876d0-764d-11ef-bceb-c7b262605968","modified":"2024-09-19T06:09:23.791881Z","created":"2024-09-19T06:09:23.791881Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi053","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T06:11:20.736 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 1 2024-09-19T06:11:20.737 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.736+0000 7f271c767640 1 -- 172.21.15.155:0/2894730809 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f27000392d0 msgr2=0x7f270003b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:20.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.736+0000 7f271c767640 1 --2- 172.21.15.155:0/2894730809 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f27000392d0 0x7f270003b790 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f27080098c0 tx=0x7f2708006db0 comp rx=0 tx=0).stop 2024-09-19T06:11:20.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.736+0000 7f271c767640 1 -- 172.21.15.155:0/2894730809 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2714100540 msgr2=0x7f2714196cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:20.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.736+0000 7f271c767640 1 --2- 172.21.15.155:0/2894730809 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2714100540 0x7f2714196cc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f27100046e0 tx=0x7f2710004710 comp rx=0 tx=0).stop 2024-09-19T06:11:20.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.736+0000 7f271c767640 1 -- 172.21.15.155:0/2894730809 shutdown_connections 2024-09-19T06:11:20.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.736+0000 7f271c767640 1 --2- 172.21.15.155:0/2894730809 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f27000392d0 0x7f270003b790 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:20.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.736+0000 7f271c767640 1 --2- 172.21.15.155:0/2894730809 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2714100540 0x7f2714196cc0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:20.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.736+0000 7f271c767640 1 -- 172.21.15.155:0/2894730809 >> 172.21.15.155:0/2894730809 conn(0x7f27140fbcd0 msgr2=0x7f2714069b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:20.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.736+0000 7f271c767640 1 -- 172.21.15.155:0/2894730809 shutdown_connections 2024-09-19T06:11:20.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:20.736+0000 7f271c767640 1 -- 172.21.15.155:0/2894730809 wait complete. 2024-09-19T06:11:21.588 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:21 smithi053 ceph-mon[27968]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:21.588 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:21 smithi053 ceph-mon[27968]: from='client.? 172.21.15.155:0/2894730809' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:22.304 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-19T06:11:22.304 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T06:11:22.547 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:11:22.609 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:11:22.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:22.978+0000 7f15ee87c640 1 -- 172.21.15.155:0/2660572950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15e81004e0 msgr2=0x7f15e81008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:22.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:22.978+0000 7f15ee87c640 1 --2- 172.21.15.155:0/2660572950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15e81004e0 0x7f15e81008e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f15d8009880 tx=0x7f15d802a5d0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:22.978+0000 7f15ee87c640 1 -- 172.21.15.155:0/2660572950 shutdown_connections 2024-09-19T06:11:22.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:22.978+0000 7f15ee87c640 1 --2- 172.21.15.155:0/2660572950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15e81004e0 0x7f15e81008e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:22.978+0000 7f15ee87c640 1 -- 172.21.15.155:0/2660572950 >> 172.21.15.155:0/2660572950 conn(0x7f15e80fbc90 msgr2=0x7f15e80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:22.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:22.978+0000 7f15ee87c640 1 -- 172.21.15.155:0/2660572950 shutdown_connections 2024-09-19T06:11:22.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:22.978+0000 7f15ee87c640 1 -- 172.21.15.155:0/2660572950 wait complete. 2024-09-19T06:11:22.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:22.979+0000 7f15ee87c640 1 Processor -- start 2024-09-19T06:11:22.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:22.979+0000 7f15ee87c640 1 -- start start 2024-09-19T06:11:22.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:22.979+0000 7f15ee87c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15e81004e0 0x7f15e8198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:22.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:22.979+0000 7f15ee87c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f15e81064a0 con 0x7f15e81004e0 2024-09-19T06:11:22.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:22.979+0000 7f15e7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15e81004e0 0x7f15e8198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:22.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:22.979+0000 7f15e7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15e81004e0 0x7f15e8198e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.155:35924/0 (socket says 172.21.15.155:35924) 2024-09-19T06:11:22.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:22.979+0000 7f15e7fff640 1 -- 172.21.15.155:0/3429379745 learned_addr learned my addr 172.21.15.155:0/3429379745 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-19T06:11:22.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:22.980+0000 7f15e7fff640 1 -- 172.21.15.155:0/3429379745 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e81993b0 con 0x7f15e81004e0 2024-09-19T06:11:22.983 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:22.980+0000 7f15e7fff640 1 --2- 172.21.15.155:0/3429379745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15e81004e0 0x7f15e8198e70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f15d8005ab0 tx=0x7f15d80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:22.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:22.980+0000 7f15e57fa640 1 -- 172.21.15.155:0/3429379745 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f15d803f070 con 0x7f15e81004e0 2024-09-19T06:11:22.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:22.980+0000 7f15e57fa640 1 -- 172.21.15.155:0/3429379745 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15d8003aa0 con 0x7f15e81004e0 2024-09-19T06:11:22.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:22.980+0000 7f15ee87c640 1 -- 172.21.15.155:0/3429379745 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e8199640 con 0x7f15e81004e0 2024-09-19T06:11:22.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:22.980+0000 7f15e57fa640 1 -- 172.21.15.155:0/3429379745 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f15d8035030 con 0x7f15e81004e0 2024-09-19T06:11:22.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:22.980+0000 7f15ee87c640 1 -- 172.21.15.155:0/3429379745 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e819c330 con 0x7f15e81004e0 2024-09-19T06:11:22.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:22.981+0000 7f15e57fa640 1 -- 172.21.15.155:0/3429379745 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46532+0+0 (secure 0 0 0) 0x7f15d8042070 con 0x7f15e81004e0 2024-09-19T06:11:22.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:22.981+0000 7f15ee87c640 1 -- 172.21.15.155:0/3429379745 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15e8106c20 con 0x7f15e81004e0 2024-09-19T06:11:22.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:22.981+0000 7f15e57fa640 1 --2- 172.21.15.155:0/3429379745 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f15d00392d0 0x7f15d003b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:22.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:22.981+0000 7f15e57fa640 1 -- 172.21.15.155:0/3429379745 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f15d806d130 con 0x7f15e81004e0 2024-09-19T06:11:22.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:22.981+0000 7f15e77fe640 1 --2- 172.21.15.155:0/3429379745 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f15d00392d0 0x7f15d003b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:22.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:22.982+0000 7f15e77fe640 1 --2- 172.21.15.155:0/3429379745 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f15d00392d0 0x7f15d003b790 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f15dc0098c0 tx=0x7f15dc006db0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:22.988 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:22.986+0000 7f15e57fa640 1 -- 172.21.15.155:0/3429379745 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f15d8040260 con 0x7f15e81004e0 2024-09-19T06:11:23.218 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:23.217+0000 7f15ee87c640 1 -- 172.21.15.155:0/3429379745 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f15e802d7c0 con 0x7f15e81004e0 2024-09-19T06:11:23.219 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:23.218+0000 7f15e57fa640 1 -- 172.21.15.155:0/3429379745 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+753 (secure 0 0 0) 0x7f15d806f0e0 con 0x7f15e81004e0 2024-09-19T06:11:23.220 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:11:23.220 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":1,"fsid":"9f2876d0-764d-11ef-bceb-c7b262605968","modified":"2024-09-19T06:09:23.791881Z","created":"2024-09-19T06:09:23.791881Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi053","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T06:11:23.220 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 1 2024-09-19T06:11:23.222 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:23.220+0000 7f15ee87c640 1 -- 172.21.15.155:0/3429379745 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f15d00392d0 msgr2=0x7f15d003b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:23.222 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:23.220+0000 7f15ee87c640 1 --2- 172.21.15.155:0/3429379745 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f15d00392d0 0x7f15d003b790 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f15dc0098c0 tx=0x7f15dc006db0 comp rx=0 tx=0).stop 2024-09-19T06:11:23.222 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:23.220+0000 7f15ee87c640 1 -- 172.21.15.155:0/3429379745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15e81004e0 msgr2=0x7f15e8198e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:23.222 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:23.220+0000 7f15ee87c640 1 --2- 172.21.15.155:0/3429379745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15e81004e0 0x7f15e8198e70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f15d8005ab0 tx=0x7f15d80048f0 comp rx=0 tx=0).stop 2024-09-19T06:11:23.222 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:23.220+0000 7f15ee87c640 1 -- 172.21.15.155:0/3429379745 shutdown_connections 2024-09-19T06:11:23.222 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:23.220+0000 7f15ee87c640 1 --2- 172.21.15.155:0/3429379745 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7f15d00392d0 0x7f15d003b790 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:23.222 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:23.220+0000 7f15ee87c640 1 --2- 172.21.15.155:0/3429379745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15e81004e0 0x7f15e8198e70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:23.222 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:23.220+0000 7f15ee87c640 1 -- 172.21.15.155:0/3429379745 >> 172.21.15.155:0/3429379745 conn(0x7f15e80fbc90 msgr2=0x7f15e818ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:23.222 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:23.220+0000 7f15ee87c640 1 -- 172.21.15.155:0/3429379745 shutdown_connections 2024-09-19T06:11:23.222 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:23.220+0000 7f15ee87c640 1 -- 172.21.15.155:0/3429379745 wait complete. 2024-09-19T06:11:24.007 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:23 smithi053 ceph-mon[27968]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:24.007 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:23 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:24.007 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:23 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:24.007 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:23 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:24.007 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:23 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-19T06:11:24.008 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:23 smithi053 ceph-mon[27968]: from='client.? 172.21.15.155:0/3429379745' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:24.834 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-19T06:11:24.835 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T06:11:25.004 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:24 smithi053 ceph-mon[27968]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-19T06:11:25.004 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:24 smithi053 ceph-mon[27968]: mgrmap e14: smithi053.trakvg(active, since 53s) 2024-09-19T06:11:25.080 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:11:25.126 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:11:25.590 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.549+0000 7fe6049f1640 1 -- 172.21.15.155:0/3201989375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6001020a0 msgr2=0x7fe600104490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:25.590 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.549+0000 7fe6049f1640 1 --2- 172.21.15.155:0/3201989375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6001020a0 0x7fe600104490 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0009880 tx=0x7fe5f002a5d0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.590 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.549+0000 7fe6049f1640 1 -- 172.21.15.155:0/3201989375 shutdown_connections 2024-09-19T06:11:25.591 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.549+0000 7fe6049f1640 1 --2- 172.21.15.155:0/3201989375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6001020a0 0x7fe600104490 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.591 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.549+0000 7fe6049f1640 1 -- 172.21.15.155:0/3201989375 >> 172.21.15.155:0/3201989375 conn(0x7fe6000fbc80 msgr2=0x7fe6000fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:25.591 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.550+0000 7fe6049f1640 1 -- 172.21.15.155:0/3201989375 shutdown_connections 2024-09-19T06:11:25.591 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.550+0000 7fe6049f1640 1 -- 172.21.15.155:0/3201989375 wait complete. 2024-09-19T06:11:25.591 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.550+0000 7fe6049f1640 1 Processor -- start 2024-09-19T06:11:25.591 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.550+0000 7fe6049f1640 1 -- start start 2024-09-19T06:11:25.591 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.551+0000 7fe6049f1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6001020a0 0x7fe600196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:25.591 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.551+0000 7fe6049f1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe600107560 con 0x7fe6001020a0 2024-09-19T06:11:25.591 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.551+0000 7fe5fe59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6001020a0 0x7fe600196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:25.591 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.551+0000 7fe5fe59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6001020a0 0x7fe600196cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.155:35954/0 (socket says 172.21.15.155:35954) 2024-09-19T06:11:25.591 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.551+0000 7fe5fe59c640 1 -- 172.21.15.155:0/37779705 learned_addr learned my addr 172.21.15.155:0/37779705 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-19T06:11:25.591 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.551+0000 7fe5fe59c640 1 -- 172.21.15.155:0/37779705 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe600197200 con 0x7fe6001020a0 2024-09-19T06:11:25.591 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.551+0000 7fe5fe59c640 1 --2- 172.21.15.155:0/37779705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6001020a0 0x7fe600196cc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0006010 tx=0x7fe5f00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:25.592 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.551+0000 7fe5ef7fe640 1 -- 172.21.15.155:0/37779705 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fe5f003f070 con 0x7fe6001020a0 2024-09-19T06:11:25.592 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.552+0000 7fe5ef7fe640 1 -- 172.21.15.155:0/37779705 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5f0003aa0 con 0x7fe6001020a0 2024-09-19T06:11:25.592 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.552+0000 7fe6049f1640 1 -- 172.21.15.155:0/37779705 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6001974f0 con 0x7fe6001020a0 2024-09-19T06:11:25.592 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.552+0000 7fe5ef7fe640 1 -- 172.21.15.155:0/37779705 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fe5f0035030 con 0x7fe6001020a0 2024-09-19T06:11:25.592 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.552+0000 7fe6049f1640 1 -- 172.21.15.155:0/37779705 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe60019a180 con 0x7fe6001020a0 2024-09-19T06:11:25.592 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.552+0000 7fe5ef7fe640 1 -- 172.21.15.155:0/37779705 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 14) ==== 46546+0+0 (secure 0 0 0) 0x7fe5f0042070 con 0x7fe6001020a0 2024-09-19T06:11:25.592 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.552+0000 7fe6049f1640 1 -- 172.21.15.155:0/37779705 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe600107ce0 con 0x7fe6001020a0 2024-09-19T06:11:25.592 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.553+0000 7fe5ef7fe640 1 --2- 172.21.15.155:0/37779705 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fe5e8039330 0x7fe5e803b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:25.592 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.553+0000 7fe5ef7fe640 1 -- 172.21.15.155:0/37779705 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe5f006d0e0 con 0x7fe6001020a0 2024-09-19T06:11:25.592 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.553+0000 7fe5fdd9b640 1 -- 172.21.15.155:0/37779705 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fe5e8039330 msgr2=0x7fe5e803b7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/3684090301 2024-09-19T06:11:25.592 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.553+0000 7fe5fdd9b640 1 --2- 172.21.15.155:0/37779705 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fe5e8039330 0x7fe5e803b7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T06:11:25.592 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.558+0000 7fe5ef7fe640 1 -- 172.21.15.155:0/37779705 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe5f0005590 con 0x7fe6001020a0 2024-09-19T06:11:25.756 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.754+0000 7fe5fdd9b640 1 -- 172.21.15.155:0/37779705 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fe5e8039330 msgr2=0x7fe5e803b7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/3684090301 2024-09-19T06:11:25.757 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.754+0000 7fe5fdd9b640 1 --2- 172.21.15.155:0/37779705 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fe5e8039330 0x7fe5e803b7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T06:11:25.794 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.791+0000 7fe6049f1640 1 -- 172.21.15.155:0/37779705 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe60002d940 con 0x7fe6001020a0 2024-09-19T06:11:25.794 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.792+0000 7fe5ef7fe640 1 -- 172.21.15.155:0/37779705 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+753 (secure 0 0 0) 0x7fe5f0040260 con 0x7fe6001020a0 2024-09-19T06:11:25.795 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:11:25.795 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":1,"fsid":"9f2876d0-764d-11ef-bceb-c7b262605968","modified":"2024-09-19T06:09:23.791881Z","created":"2024-09-19T06:09:23.791881Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi053","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T06:11:25.795 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 1 2024-09-19T06:11:25.797 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.794+0000 7fe6049f1640 1 -- 172.21.15.155:0/37779705 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fe5e8039330 msgr2=0x7fe5e803b7f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T06:11:25.797 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.794+0000 7fe6049f1640 1 --2- 172.21.15.155:0/37779705 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fe5e8039330 0x7fe5e803b7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.797 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.794+0000 7fe6049f1640 1 -- 172.21.15.155:0/37779705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6001020a0 msgr2=0x7fe600196cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:25.797 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.794+0000 7fe6049f1640 1 --2- 172.21.15.155:0/37779705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6001020a0 0x7fe600196cc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0006010 tx=0x7fe5f00048f0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.797 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.794+0000 7fe6049f1640 1 -- 172.21.15.155:0/37779705 shutdown_connections 2024-09-19T06:11:25.797 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.794+0000 7fe6049f1640 1 --2- 172.21.15.155:0/37779705 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fe5e8039330 0x7fe5e803b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.797 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.794+0000 7fe6049f1640 1 --2- 172.21.15.155:0/37779705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6001020a0 0x7fe600196cc0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.797 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.794+0000 7fe6049f1640 1 -- 172.21.15.155:0/37779705 >> 172.21.15.155:0/37779705 conn(0x7fe6000fbc80 msgr2=0x7fe6000fc770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:25.797 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.794+0000 7fe6049f1640 1 -- 172.21.15.155:0/37779705 shutdown_connections 2024-09-19T06:11:25.797 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:25.794+0000 7fe6049f1640 1 -- 172.21.15.155:0/37779705 wait complete. 2024-09-19T06:11:26.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:25 smithi053 ceph-mon[27968]: from='client.? 172.21.15.155:0/37779705' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:27.355 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-19T06:11:27.356 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T06:11:27.603 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:11:27.648 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:11:28.041 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.038+0000 7fe4413e9640 1 -- 172.21.15.155:0/1058159542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe43c102070 msgr2=0x7fe43c104460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:28.041 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.038+0000 7fe4413e9640 1 --2- 172.21.15.155:0/1058159542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe43c102070 0x7fe43c104460 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe42c009880 tx=0x7fe42c02a5d0 comp rx=0 tx=0).stop 2024-09-19T06:11:28.041 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.038+0000 7fe4413e9640 1 -- 172.21.15.155:0/1058159542 shutdown_connections 2024-09-19T06:11:28.041 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.038+0000 7fe4413e9640 1 --2- 172.21.15.155:0/1058159542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe43c102070 0x7fe43c104460 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:28.041 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.038+0000 7fe4413e9640 1 -- 172.21.15.155:0/1058159542 >> 172.21.15.155:0/1058159542 conn(0x7fe43c0fbc90 msgr2=0x7fe43c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:28.041 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.038+0000 7fe4413e9640 1 -- 172.21.15.155:0/1058159542 shutdown_connections 2024-09-19T06:11:28.042 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.038+0000 7fe4413e9640 1 -- 172.21.15.155:0/1058159542 wait complete. 2024-09-19T06:11:28.042 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.039+0000 7fe4413e9640 1 Processor -- start 2024-09-19T06:11:28.044 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.039+0000 7fe4413e9640 1 -- start start 2024-09-19T06:11:28.044 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.039+0000 7fe4413e9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe43c102070 0x7fe43c196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:28.044 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.039+0000 7fe4413e9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c107530 con 0x7fe43c102070 2024-09-19T06:11:28.044 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.039+0000 7fe43affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe43c102070 0x7fe43c196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:28.044 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.039+0000 7fe43affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe43c102070 0x7fe43c196c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.155:35972/0 (socket says 172.21.15.155:35972) 2024-09-19T06:11:28.044 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.039+0000 7fe43affd640 1 -- 172.21.15.155:0/237523663 learned_addr learned my addr 172.21.15.155:0/237523663 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-19T06:11:28.045 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.039+0000 7fe43affd640 1 -- 172.21.15.155:0/237523663 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe43c197140 con 0x7fe43c102070 2024-09-19T06:11:28.045 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.040+0000 7fe43affd640 1 --2- 172.21.15.155:0/237523663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe43c102070 0x7fe43c196c00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe42c005ab0 tx=0x7fe42c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:28.045 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.040+0000 7fe423fff640 1 -- 172.21.15.155:0/237523663 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fe42c03f070 con 0x7fe43c102070 2024-09-19T06:11:28.045 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.040+0000 7fe4413e9640 1 -- 172.21.15.155:0/237523663 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe43c197430 con 0x7fe43c102070 2024-09-19T06:11:28.045 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.040+0000 7fe423fff640 1 -- 172.21.15.155:0/237523663 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe42c003aa0 con 0x7fe43c102070 2024-09-19T06:11:28.045 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.040+0000 7fe423fff640 1 -- 172.21.15.155:0/237523663 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fe42c035030 con 0x7fe43c102070 2024-09-19T06:11:28.045 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.040+0000 7fe4413e9640 1 -- 172.21.15.155:0/237523663 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe43c19a0c0 con 0x7fe43c102070 2024-09-19T06:11:28.045 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.041+0000 7fe423fff640 1 -- 172.21.15.155:0/237523663 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 14) ==== 46546+0+0 (secure 0 0 0) 0x7fe42c042070 con 0x7fe43c102070 2024-09-19T06:11:28.045 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.041+0000 7fe4413e9640 1 -- 172.21.15.155:0/237523663 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe43c107cb0 con 0x7fe43c102070 2024-09-19T06:11:28.045 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.041+0000 7fe423fff640 1 --2- 172.21.15.155:0/237523663 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fe424039330 0x7fe42403b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:28.045 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.041+0000 7fe423fff640 1 -- 172.21.15.155:0/237523663 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe42c06d230 con 0x7fe43c102070 2024-09-19T06:11:28.045 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.041+0000 7fe43a7fc640 1 -- 172.21.15.155:0/237523663 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fe424039330 msgr2=0x7fe42403b7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/3684090301 2024-09-19T06:11:28.046 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.041+0000 7fe43a7fc640 1 --2- 172.21.15.155:0/237523663 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fe424039330 0x7fe42403b7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T06:11:28.048 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.046+0000 7fe423fff640 1 -- 172.21.15.155:0/237523663 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe42c003cc0 con 0x7fe43c102070 2024-09-19T06:11:28.245 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.242+0000 7fe43a7fc640 1 -- 172.21.15.155:0/237523663 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fe424039330 msgr2=0x7fe42403b7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/3684090301 2024-09-19T06:11:28.245 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.242+0000 7fe43a7fc640 1 --2- 172.21.15.155:0/237523663 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fe424039330 0x7fe42403b7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T06:11:28.282 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.280+0000 7fe4413e9640 1 -- 172.21.15.155:0/237523663 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe43c02d8c0 con 0x7fe43c102070 2024-09-19T06:11:28.283 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.280+0000 7fe423fff640 1 -- 172.21.15.155:0/237523663 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+753 (secure 0 0 0) 0x7fe42c03b4b0 con 0x7fe43c102070 2024-09-19T06:11:28.283 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:11:28.283 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":1,"fsid":"9f2876d0-764d-11ef-bceb-c7b262605968","modified":"2024-09-19T06:09:23.791881Z","created":"2024-09-19T06:09:23.791881Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi053","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T06:11:28.283 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 1 2024-09-19T06:11:28.285 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.282+0000 7fe4413e9640 1 -- 172.21.15.155:0/237523663 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fe424039330 msgr2=0x7fe42403b7f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T06:11:28.285 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.282+0000 7fe4413e9640 1 --2- 172.21.15.155:0/237523663 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fe424039330 0x7fe42403b7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:28.285 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.282+0000 7fe4413e9640 1 -- 172.21.15.155:0/237523663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe43c102070 msgr2=0x7fe43c196c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:28.285 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.282+0000 7fe4413e9640 1 --2- 172.21.15.155:0/237523663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe43c102070 0x7fe43c196c00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe42c005ab0 tx=0x7fe42c0048f0 comp rx=0 tx=0).stop 2024-09-19T06:11:28.286 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.283+0000 7fe4413e9640 1 -- 172.21.15.155:0/237523663 shutdown_connections 2024-09-19T06:11:28.286 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.283+0000 7fe4413e9640 1 --2- 172.21.15.155:0/237523663 >> [v2:172.21.15.53:6800/3684090301,v1:172.21.15.53:6801/3684090301] conn(0x7fe424039330 0x7fe42403b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:28.286 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.283+0000 7fe4413e9640 1 --2- 172.21.15.155:0/237523663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe43c102070 0x7fe43c196c00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:28.286 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.283+0000 7fe4413e9640 1 -- 172.21.15.155:0/237523663 >> 172.21.15.155:0/237523663 conn(0x7fe43c0fbc90 msgr2=0x7fe43c0fc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:28.286 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.283+0000 7fe4413e9640 1 -- 172.21.15.155:0/237523663 shutdown_connections 2024-09-19T06:11:28.286 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:28.283+0000 7fe4413e9640 1 -- 172.21.15.155:0/237523663 wait complete. 2024-09-19T06:11:28.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:28 smithi053 ceph-mon[27968]: from='client.? 172.21.15.155:0/237523663' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:29.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:29 smithi053 ceph-mon[27968]: Active manager daemon smithi053.trakvg restarted 2024-09-19T06:11:29.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:29 smithi053 ceph-mon[27968]: Activating manager daemon smithi053.trakvg 2024-09-19T06:11:29.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:29 smithi053 ceph-mon[27968]: osdmap e5: 0 total, 0 up, 0 in 2024-09-19T06:11:29.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:29 smithi053 ceph-mon[27968]: mgrmap e15: smithi053.trakvg(active, starting, since 0.00322326s) 2024-09-19T06:11:29.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata", "id": "smithi053"}]: dispatch 2024-09-19T06:11:29.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mgr metadata", "who": "smithi053.trakvg", "id": "smithi053.trakvg"}]: dispatch 2024-09-19T06:11:29.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T06:11:29.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T06:11:29.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T06:11:29.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:29 smithi053 ceph-mon[27968]: Manager daemon smithi053.trakvg is now available 2024-09-19T06:11:29.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:11:29.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:29.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:11:29.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi053.trakvg/mirror_snapshot_schedule"}]: dispatch 2024-09-19T06:11:29.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi053.trakvg/trash_purge_schedule"}]: dispatch 2024-09-19T06:11:29.911 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-19T06:11:29.912 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T06:11:30.146 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:11:30.190 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:11:30.627 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.616+0000 7f9ae2e3f640 1 -- 172.21.15.155:0/1757861229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9adc100540 msgr2=0x7f9adc100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:30.627 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.616+0000 7f9ae2e3f640 1 --2- 172.21.15.155:0/1757861229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9adc100540 0x7f9adc100940 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0009880 tx=0x7f9ad002a5d0 comp rx=0 tx=0).stop 2024-09-19T06:11:30.628 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.616+0000 7f9ae2e3f640 1 -- 172.21.15.155:0/1757861229 shutdown_connections 2024-09-19T06:11:30.628 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.616+0000 7f9ae2e3f640 1 --2- 172.21.15.155:0/1757861229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9adc100540 0x7f9adc100940 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:30.628 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.616+0000 7f9ae2e3f640 1 -- 172.21.15.155:0/1757861229 >> 172.21.15.155:0/1757861229 conn(0x7f9adc0fbcd0 msgr2=0x7f9adc0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:30.628 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.616+0000 7f9ae2e3f640 1 -- 172.21.15.155:0/1757861229 shutdown_connections 2024-09-19T06:11:30.628 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.616+0000 7f9ae2e3f640 1 -- 172.21.15.155:0/1757861229 wait complete. 2024-09-19T06:11:30.628 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.616+0000 7f9ae2e3f640 1 Processor -- start 2024-09-19T06:11:30.628 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.617+0000 7f9ae2e3f640 1 -- start start 2024-09-19T06:11:30.628 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.617+0000 7f9ae2e3f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9adc100540 0x7f9adc195bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:30.628 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.617+0000 7f9ae2e3f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9adc1064a0 con 0x7f9adc100540 2024-09-19T06:11:30.628 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.617+0000 7f9ae0bdb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9adc100540 0x7f9adc195bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:30.628 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.617+0000 7f9ae0bdb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9adc100540 0x7f9adc195bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.155:52204/0 (socket says 172.21.15.155:52204) 2024-09-19T06:11:30.628 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.617+0000 7f9ae0bdb640 1 -- 172.21.15.155:0/1576741719 learned_addr learned my addr 172.21.15.155:0/1576741719 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-19T06:11:30.629 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.617+0000 7f9ae0bdb640 1 -- 172.21.15.155:0/1576741719 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9adc196100 con 0x7f9adc100540 2024-09-19T06:11:30.629 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.617+0000 7f9ae0bdb640 1 --2- 172.21.15.155:0/1576741719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9adc100540 0x7f9adc195bc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0005ab0 tx=0x7f9ad00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:30.629 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.618+0000 7f9ad9ffb640 1 -- 172.21.15.155:0/1576741719 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f9ad003f070 con 0x7f9adc100540 2024-09-19T06:11:30.629 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.618+0000 7f9ad9ffb640 1 -- 172.21.15.155:0/1576741719 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ad0003aa0 con 0x7f9adc100540 2024-09-19T06:11:30.629 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.618+0000 7f9ae2e3f640 1 -- 172.21.15.155:0/1576741719 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9adc196390 con 0x7f9adc100540 2024-09-19T06:11:30.629 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.618+0000 7f9ad9ffb640 1 -- 172.21.15.155:0/1576741719 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f9ad0035030 con 0x7f9adc100540 2024-09-19T06:11:30.629 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.618+0000 7f9ae2e3f640 1 -- 172.21.15.155:0/1576741719 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9adc199080 con 0x7f9adc100540 2024-09-19T06:11:30.629 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.619+0000 7f9ad9ffb640 1 -- 172.21.15.155:0/1576741719 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 46397+0+0 (secure 0 0 0) 0x7f9ad0042070 con 0x7f9adc100540 2024-09-19T06:11:30.629 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.619+0000 7f9ae2e3f640 1 -- 172.21.15.155:0/1576741719 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9adc106c20 con 0x7f9adc100540 2024-09-19T06:11:30.629 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.619+0000 7f9ad9ffb640 1 --2- 172.21.15.155:0/1576741719 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f9acc039170 0x7f9acc03b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:30.629 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.619+0000 7f9ad9ffb640 1 -- 172.21.15.155:0/1576741719 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f9ad006cf80 con 0x7f9adc100540 2024-09-19T06:11:30.629 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.620+0000 7f9adbfff640 1 --2- 172.21.15.155:0/1576741719 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f9acc039170 0x7f9acc03b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:30.629 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.620+0000 7f9adbfff640 1 --2- 172.21.15.155:0/1576741719 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f9acc039170 0x7f9acc03b630 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9ac80098c0 tx=0x7f9ac8006db0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:30.629 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.624+0000 7f9ad9ffb640 1 -- 172.21.15.155:0/1576741719 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9ad002f300 con 0x7f9adc100540 2024-09-19T06:11:30.773 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:30 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:30.773 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:30 smithi053 ceph-mon[27968]: mgrmap e16: smithi053.trakvg(active, since 1.00536s) 2024-09-19T06:11:30.773 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:30 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:30.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.889+0000 7f9ae2e3f640 1 -- 172.21.15.155:0/1576741719 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9adc02d7c0 con 0x7f9adc100540 2024-09-19T06:11:30.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.890+0000 7f9ad9ffb640 1 -- 172.21.15.155:0/1576741719 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+753 (secure 0 0 0) 0x7f9ad003b4b0 con 0x7f9adc100540 2024-09-19T06:11:30.893 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:11:30.893 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":1,"fsid":"9f2876d0-764d-11ef-bceb-c7b262605968","modified":"2024-09-19T06:09:23.791881Z","created":"2024-09-19T06:09:23.791881Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi053","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T06:11:30.893 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 1 2024-09-19T06:11:30.895 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.892+0000 7f9ae2e3f640 1 -- 172.21.15.155:0/1576741719 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f9acc039170 msgr2=0x7f9acc03b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:30.895 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.892+0000 7f9ae2e3f640 1 --2- 172.21.15.155:0/1576741719 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f9acc039170 0x7f9acc03b630 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9ac80098c0 tx=0x7f9ac8006db0 comp rx=0 tx=0).stop 2024-09-19T06:11:30.895 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.892+0000 7f9ae2e3f640 1 -- 172.21.15.155:0/1576741719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9adc100540 msgr2=0x7f9adc195bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:30.895 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.892+0000 7f9ae2e3f640 1 --2- 172.21.15.155:0/1576741719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9adc100540 0x7f9adc195bc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0005ab0 tx=0x7f9ad00048f0 comp rx=0 tx=0).stop 2024-09-19T06:11:30.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.892+0000 7f9ae2e3f640 1 -- 172.21.15.155:0/1576741719 shutdown_connections 2024-09-19T06:11:30.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.892+0000 7f9ae2e3f640 1 --2- 172.21.15.155:0/1576741719 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f9acc039170 0x7f9acc03b630 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:30.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.892+0000 7f9ae2e3f640 1 --2- 172.21.15.155:0/1576741719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9adc100540 0x7f9adc195bc0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:30.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.892+0000 7f9ae2e3f640 1 -- 172.21.15.155:0/1576741719 >> 172.21.15.155:0/1576741719 conn(0x7f9adc0fbcd0 msgr2=0x7f9adc0fc760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:30.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.893+0000 7f9ae2e3f640 1 -- 172.21.15.155:0/1576741719 shutdown_connections 2024-09-19T06:11:30.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:30.893+0000 7f9ae2e3f640 1 -- 172.21.15.155:0/1576741719 wait complete. 2024-09-19T06:11:31.568 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:31 smithi053 ceph-mon[27968]: [19/Sep/2024:06:11:30] ENGINE Bus STARTING 2024-09-19T06:11:31.569 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:31 smithi053 ceph-mon[27968]: from='client.? 172.21.15.155:0/1576741719' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:31.569 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:31 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:32.536 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-19T06:11:32.536 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T06:11:32.779 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:32 smithi053 ceph-mon[27968]: [19/Sep/2024:06:11:30] ENGINE Serving on https://172.21.15.53:7150 2024-09-19T06:11:32.779 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:32 smithi053 ceph-mon[27968]: [19/Sep/2024:06:11:30] ENGINE Error in HTTPServer.serve 2024-09-19T06:11:32.779 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Traceback (most recent call last): 2024-09-19T06:11:32.779 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T06:11:32.780 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: self._connections.run(self.expiration_interval) 2024-09-19T06:11:32.780 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T06:11:32.780 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: self._run(expiration_interval) 2024-09-19T06:11:32.780 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T06:11:32.780 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T06:11:32.780 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T06:11:32.780 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T06:11:32.780 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T06:11:32.780 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: s = self.context.wrap_socket( 2024-09-19T06:11:32.781 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T06:11:32.781 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: return self.sslsocket_class._create( 2024-09-19T06:11:32.781 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T06:11:32.781 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: self.do_handshake() 2024-09-19T06:11:32.781 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T06:11:32.781 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: self._sslobj.do_handshake() 2024-09-19T06:11:32.781 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T06:11:32.781 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:32 smithi053 ceph-mon[27968]: [19/Sep/2024:06:11:30] ENGINE Serving on http://172.21.15.53:8765 2024-09-19T06:11:32.781 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:32 smithi053 ceph-mon[27968]: [19/Sep/2024:06:11:30] ENGINE Bus STARTED 2024-09-19T06:11:32.781 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:32 smithi053 ceph-mon[27968]: mgrmap e17: smithi053.trakvg(active, since 2s) 2024-09-19T06:11:33.234 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:11:33.288 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:11:33.837 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:33.756+0000 7feaec005640 1 -- 172.21.15.155:0/3972579178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feae41020a0 msgr2=0x7feae4104490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:33.837 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:33.756+0000 7feaec005640 1 --2- 172.21.15.155:0/3972579178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feae41020a0 0x7feae4104490 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fead4009880 tx=0x7fead402a5d0 comp rx=0 tx=0).stop 2024-09-19T06:11:33.837 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:33.756+0000 7feaec005640 1 -- 172.21.15.155:0/3972579178 shutdown_connections 2024-09-19T06:11:33.837 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:33.756+0000 7feaec005640 1 --2- 172.21.15.155:0/3972579178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feae41020a0 0x7feae4104490 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:33.837 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:33.756+0000 7feaec005640 1 -- 172.21.15.155:0/3972579178 >> 172.21.15.155:0/3972579178 conn(0x7feae40fbc80 msgr2=0x7feae40fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:33.837 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:33.756+0000 7feaec005640 1 -- 172.21.15.155:0/3972579178 shutdown_connections 2024-09-19T06:11:33.837 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:33.756+0000 7feaec005640 1 -- 172.21.15.155:0/3972579178 wait complete. 2024-09-19T06:11:33.837 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:33.757+0000 7feaec005640 1 Processor -- start 2024-09-19T06:11:33.837 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:33.757+0000 7feaec005640 1 -- start start 2024-09-19T06:11:33.838 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:33.757+0000 7feaec005640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feae41020a0 0x7feae4196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:33.838 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:33.757+0000 7feaec005640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7feae4107560 con 0x7feae41020a0 2024-09-19T06:11:33.838 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:33.757+0000 7feae9da1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feae41020a0 0x7feae4196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:33.838 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:33.757+0000 7feae9da1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feae41020a0 0x7feae4196cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.155:52222/0 (socket says 172.21.15.155:52222) 2024-09-19T06:11:33.838 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:33.757+0000 7feae9da1640 1 -- 172.21.15.155:0/915497427 learned_addr learned my addr 172.21.15.155:0/915497427 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-19T06:11:33.838 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:33.758+0000 7feae9da1640 1 -- 172.21.15.155:0/915497427 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae4197200 con 0x7feae41020a0 2024-09-19T06:11:33.838 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:33.758+0000 7feae9da1640 1 --2- 172.21.15.155:0/915497427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feae41020a0 0x7feae4196cc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fead4006010 tx=0x7fead40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:33.838 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:33.758+0000 7feadaffd640 1 -- 172.21.15.155:0/915497427 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fead403f070 con 0x7feae41020a0 2024-09-19T06:11:33.838 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:33.758+0000 7feadaffd640 1 -- 172.21.15.155:0/915497427 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fead4003aa0 con 0x7feae41020a0 2024-09-19T06:11:33.838 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:33.758+0000 7feaec005640 1 -- 172.21.15.155:0/915497427 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feae41974f0 con 0x7feae41020a0 2024-09-19T06:11:33.838 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:33.758+0000 7feadaffd640 1 -- 172.21.15.155:0/915497427 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fead4035030 con 0x7feae41020a0 2024-09-19T06:11:33.838 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:33.758+0000 7feaec005640 1 -- 172.21.15.155:0/915497427 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feae419a180 con 0x7feae41020a0 2024-09-19T06:11:33.838 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:33.759+0000 7feadaffd640 1 -- 172.21.15.155:0/915497427 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 46589+0+0 (secure 0 0 0) 0x7fead4042070 con 0x7feae41020a0 2024-09-19T06:11:33.839 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:33.759+0000 7feaec005640 1 -- 172.21.15.155:0/915497427 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feae4107ce0 con 0x7feae41020a0 2024-09-19T06:11:33.839 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:33.759+0000 7feadaffd640 1 --2- 172.21.15.155:0/915497427 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fead00393d0 0x7fead003b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:33.839 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:33.759+0000 7feadaffd640 1 -- 172.21.15.155:0/915497427 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fead406d1c0 con 0x7feae41020a0 2024-09-19T06:11:33.839 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:33.760+0000 7feae95a0640 1 --2- 172.21.15.155:0/915497427 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fead00393d0 0x7fead003b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:33.839 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:33.760+0000 7feae95a0640 1 --2- 172.21.15.155:0/915497427 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fead00393d0 0x7fead003b890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7feae00098c0 tx=0x7feae0006db0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:33.839 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:33.765+0000 7feadaffd640 1 -- 172.21.15.155:0/915497427 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fead4040260 con 0x7feae41020a0 2024-09-19T06:11:34.000 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:33.997+0000 7feaec005640 1 -- 172.21.15.155:0/915497427 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7feae402d940 con 0x7feae41020a0 2024-09-19T06:11:34.001 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:33.998+0000 7feadaffd640 1 -- 172.21.15.155:0/915497427 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+753 (secure 0 0 0) 0x7fead4005590 con 0x7feae41020a0 2024-09-19T06:11:34.004 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:11:34.004 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":1,"fsid":"9f2876d0-764d-11ef-bceb-c7b262605968","modified":"2024-09-19T06:09:23.791881Z","created":"2024-09-19T06:09:23.791881Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi053","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T06:11:34.005 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 1 2024-09-19T06:11:34.006 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:34.000+0000 7feaec005640 1 -- 172.21.15.155:0/915497427 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fead00393d0 msgr2=0x7fead003b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:34.006 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:34.000+0000 7feaec005640 1 --2- 172.21.15.155:0/915497427 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fead00393d0 0x7fead003b890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7feae00098c0 tx=0x7feae0006db0 comp rx=0 tx=0).stop 2024-09-19T06:11:34.006 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:34.000+0000 7feaec005640 1 -- 172.21.15.155:0/915497427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feae41020a0 msgr2=0x7feae4196cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:34.006 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:34.000+0000 7feaec005640 1 --2- 172.21.15.155:0/915497427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feae41020a0 0x7feae4196cc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fead4006010 tx=0x7fead40048f0 comp rx=0 tx=0).stop 2024-09-19T06:11:34.006 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:34.000+0000 7feaec005640 1 -- 172.21.15.155:0/915497427 shutdown_connections 2024-09-19T06:11:34.006 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:34.000+0000 7feaec005640 1 --2- 172.21.15.155:0/915497427 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fead00393d0 0x7fead003b890 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:34.006 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:34.000+0000 7feaec005640 1 --2- 172.21.15.155:0/915497427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feae41020a0 0x7feae4196cc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:34.006 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:34.000+0000 7feaec005640 1 -- 172.21.15.155:0/915497427 >> 172.21.15.155:0/915497427 conn(0x7feae40fbc80 msgr2=0x7feae40fc770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:34.006 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:34.000+0000 7feaec005640 1 -- 172.21.15.155:0/915497427 shutdown_connections 2024-09-19T06:11:34.006 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:34.000+0000 7feaec005640 1 -- 172.21.15.155:0/915497427 wait complete. 2024-09-19T06:11:34.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:34 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:34.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:34 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:34.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:34 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:34.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:34 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:34.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:34 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd/host:smithi053", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:11:34.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:34 smithi053 ceph-mon[27968]: from='client.? 172.21.15.155:0/915497427' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:35.774 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:35 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:35.774 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:35 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:35.774 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:35 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:35.774 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:35 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:35.775 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:35 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd/host:smithi155", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:11:35.775 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:35 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:11:35.775 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:35 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:11:35.775 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:35 smithi053 ceph-mon[27968]: Updating smithi053:/etc/ceph/ceph.conf 2024-09-19T06:11:35.775 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:35 smithi053 ceph-mon[27968]: Updating smithi155:/etc/ceph/ceph.conf 2024-09-19T06:11:35.775 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:35 smithi053 ceph-mon[27968]: Updating smithi155:/var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:11:35.775 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:35 smithi053 ceph-mon[27968]: Updating smithi053:/var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:11:35.775 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:35 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:35.775 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:35 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:35.775 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:35 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:35.775 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:35 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:35.775 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:35 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:35.776 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:35 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi155", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-19T06:11:35.776 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:35 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi155", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-19T06:11:35.776 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:35 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:11:36.111 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-19T06:11:36.111 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T06:11:36.740 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:11:36.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:36 smithi053 ceph-mon[27968]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:11:36.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:36 smithi053 ceph-mon[27968]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:11:36.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:36 smithi053 ceph-mon[27968]: Updating smithi155:/var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:11:36.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:36 smithi053 ceph-mon[27968]: Updating smithi053:/var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:11:36.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:36 smithi053 ceph-mon[27968]: Deploying daemon ceph-exporter.smithi155 on smithi155 2024-09-19T06:11:37.231 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.218+0000 7f4970420640 1 -- 172.21.15.155:0/1381974493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4968102790 msgr2=0x7f4968102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:37.231 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.218+0000 7f4970420640 1 --2- 172.21.15.155:0/1381974493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4968102790 0x7f4968102b70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4964009810 tx=0x7f496402a540 comp rx=0 tx=0).stop 2024-09-19T06:11:37.232 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.218+0000 7f4970420640 1 -- 172.21.15.155:0/1381974493 shutdown_connections 2024-09-19T06:11:37.232 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.218+0000 7f4970420640 1 --2- 172.21.15.155:0/1381974493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4968102790 0x7f4968102b70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:37.232 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.219+0000 7f4970420640 1 -- 172.21.15.155:0/1381974493 >> 172.21.15.155:0/1381974493 conn(0x7f4968077310 msgr2=0x7f49680fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:37.232 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.219+0000 7f4970420640 1 -- 172.21.15.155:0/1381974493 shutdown_connections 2024-09-19T06:11:37.232 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.219+0000 7f4970420640 1 -- 172.21.15.155:0/1381974493 wait complete. 2024-09-19T06:11:37.233 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.219+0000 7f4970420640 1 Processor -- start 2024-09-19T06:11:37.233 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.219+0000 7f4970420640 1 -- start start 2024-09-19T06:11:37.233 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.219+0000 7f4970420640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4968102790 0x7f49681003f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:37.233 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.219+0000 7f4970420640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f496810bb30 con 0x7f4968102790 2024-09-19T06:11:37.233 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.220+0000 7f496e1bc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4968102790 0x7f49681003f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:37.234 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.220+0000 7f496e1bc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4968102790 0x7f49681003f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.155:52234/0 (socket says 172.21.15.155:52234) 2024-09-19T06:11:37.234 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.220+0000 7f496e1bc640 1 -- 172.21.15.155:0/172491323 learned_addr learned my addr 172.21.15.155:0/172491323 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-19T06:11:37.234 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.220+0000 7f496e1bc640 1 -- 172.21.15.155:0/172491323 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49681022e0 con 0x7f4968102790 2024-09-19T06:11:37.234 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.220+0000 7f496e1bc640 1 --2- 172.21.15.155:0/172491323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4968102790 0x7f49681003f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f49640045e0 tx=0x7f4964004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:37.234 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.220+0000 7f495f7fe640 1 -- 172.21.15.155:0/172491323 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f496403f070 con 0x7f4968102790 2024-09-19T06:11:37.234 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.220+0000 7f495f7fe640 1 -- 172.21.15.155:0/172491323 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4964031870 con 0x7f4968102790 2024-09-19T06:11:37.235 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.220+0000 7f4970420640 1 -- 172.21.15.155:0/172491323 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4968100930 con 0x7f4968102790 2024-09-19T06:11:37.235 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.220+0000 7f495f7fe640 1 -- 172.21.15.155:0/172491323 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f4964030790 con 0x7f4968102790 2024-09-19T06:11:37.235 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.221+0000 7f4970420640 1 -- 172.21.15.155:0/172491323 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4968100df0 con 0x7f4968102790 2024-09-19T06:11:37.235 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.221+0000 7f4970420640 1 -- 172.21.15.155:0/172491323 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4968069f30 con 0x7f4968102790 2024-09-19T06:11:37.235 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.221+0000 7f495f7fe640 1 -- 172.21.15.155:0/172491323 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 46589+0+0 (secure 0 0 0) 0x7f4964004e80 con 0x7f4968102790 2024-09-19T06:11:37.236 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.222+0000 7f495f7fe640 1 --2- 172.21.15.155:0/172491323 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f49540393d0 0x7f495403b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:37.236 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.222+0000 7f495f7fe640 1 -- 172.21.15.155:0/172491323 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f496406d560 con 0x7f4968102790 2024-09-19T06:11:37.236 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.222+0000 7f496d9bb640 1 --2- 172.21.15.155:0/172491323 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f49540393d0 0x7f495403b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:37.236 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.222+0000 7f496d9bb640 1 --2- 172.21.15.155:0/172491323 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f49540393d0 0x7f495403b890 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f49580098c0 tx=0x7f4958006db0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:37.236 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.227+0000 7f495f7fe640 1 -- 172.21.15.155:0/172491323 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f49640052c0 con 0x7f4968102790 2024-09-19T06:11:37.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.467+0000 7f4970420640 1 -- 172.21.15.155:0/172491323 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f4968101920 con 0x7f4968102790 2024-09-19T06:11:37.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.467+0000 7f495f7fe640 1 -- 172.21.15.155:0/172491323 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+753 (secure 0 0 0) 0x7f49640047c0 con 0x7f4968102790 2024-09-19T06:11:37.469 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:11:37.469 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":1,"fsid":"9f2876d0-764d-11ef-bceb-c7b262605968","modified":"2024-09-19T06:09:23.791881Z","created":"2024-09-19T06:09:23.791881Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi053","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T06:11:37.470 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 1 2024-09-19T06:11:37.471 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.470+0000 7f4970420640 1 -- 172.21.15.155:0/172491323 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f49540393d0 msgr2=0x7f495403b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:37.471 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.470+0000 7f4970420640 1 --2- 172.21.15.155:0/172491323 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f49540393d0 0x7f495403b890 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f49580098c0 tx=0x7f4958006db0 comp rx=0 tx=0).stop 2024-09-19T06:11:37.471 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.470+0000 7f4970420640 1 -- 172.21.15.155:0/172491323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4968102790 msgr2=0x7f49681003f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:37.471 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.470+0000 7f4970420640 1 --2- 172.21.15.155:0/172491323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4968102790 0x7f49681003f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f49640045e0 tx=0x7f4964004610 comp rx=0 tx=0).stop 2024-09-19T06:11:37.471 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.470+0000 7f4970420640 1 -- 172.21.15.155:0/172491323 shutdown_connections 2024-09-19T06:11:37.471 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.470+0000 7f4970420640 1 --2- 172.21.15.155:0/172491323 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f49540393d0 0x7f495403b890 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:37.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.470+0000 7f4970420640 1 --2- 172.21.15.155:0/172491323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4968102790 0x7f49681003f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:37.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.470+0000 7f4970420640 1 -- 172.21.15.155:0/172491323 >> 172.21.15.155:0/172491323 conn(0x7f4968077310 msgr2=0x7f4968105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:37.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.470+0000 7f4970420640 1 -- 172.21.15.155:0/172491323 shutdown_connections 2024-09-19T06:11:37.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:37.470+0000 7f4970420640 1 -- 172.21.15.155:0/172491323 wait complete. 2024-09-19T06:11:37.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:37 smithi053 ceph-mon[27968]: from='client.? 172.21.15.155:0/172491323' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:39.113 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-19T06:11:39.113 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T06:11:39.787 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:11:40.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:39 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:40.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:39 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:40.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:39 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:40.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:39 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:40.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:39 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi155", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-19T06:11:40.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:39 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi155", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-09-19T06:11:40.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:39 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:11:40.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:39 smithi053 ceph-mon[27968]: Deploying daemon crash.smithi155 on smithi155 2024-09-19T06:11:40.209 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.206+0000 7f082a1ca640 1 -- 172.21.15.155:0/3917243050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0824107660 msgr2=0x7f0824107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:40.209 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.206+0000 7f082a1ca640 1 --2- 172.21.15.155:0/3917243050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0824107660 0x7f0824107a40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0814009880 tx=0x7f081402a5d0 comp rx=0 tx=0).stop 2024-09-19T06:11:40.209 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.207+0000 7f082a1ca640 1 -- 172.21.15.155:0/3917243050 shutdown_connections 2024-09-19T06:11:40.209 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.207+0000 7f082a1ca640 1 --2- 172.21.15.155:0/3917243050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0824107660 0x7f0824107a40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:40.209 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.207+0000 7f082a1ca640 1 -- 172.21.15.155:0/3917243050 >> 172.21.15.155:0/3917243050 conn(0x7f08240fd290 msgr2=0x7f08240ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:40.209 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.207+0000 7f082a1ca640 1 -- 172.21.15.155:0/3917243050 shutdown_connections 2024-09-19T06:11:40.209 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.207+0000 7f082a1ca640 1 -- 172.21.15.155:0/3917243050 wait complete. 2024-09-19T06:11:40.210 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.207+0000 7f082a1ca640 1 Processor -- start 2024-09-19T06:11:40.210 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.207+0000 7f082a1ca640 1 -- start start 2024-09-19T06:11:40.212 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.208+0000 7f082a1ca640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0824107660 0x7f0824197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:40.212 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.208+0000 7f082a1ca640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f082410bb40 con 0x7f0824107660 2024-09-19T06:11:40.212 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.208+0000 7f08237fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0824107660 0x7f0824197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:40.212 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.208+0000 7f08237fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0824107660 0x7f0824197020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.155:58412/0 (socket says 172.21.15.155:58412) 2024-09-19T06:11:40.212 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.208+0000 7f08237fe640 1 -- 172.21.15.155:0/2910809794 learned_addr learned my addr 172.21.15.155:0/2910809794 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-19T06:11:40.212 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.208+0000 7f08237fe640 1 -- 172.21.15.155:0/2910809794 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0824197560 con 0x7f0824107660 2024-09-19T06:11:40.212 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.208+0000 7f08237fe640 1 --2- 172.21.15.155:0/2910809794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0824107660 0x7f0824197020 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0814006010 tx=0x7f08140048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:40.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.209+0000 7f08217fa640 1 -- 172.21.15.155:0/2910809794 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f081403f070 con 0x7f0824107660 2024-09-19T06:11:40.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.209+0000 7f082a1ca640 1 -- 172.21.15.155:0/2910809794 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08241977f0 con 0x7f0824107660 2024-09-19T06:11:40.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.209+0000 7f08217fa640 1 -- 172.21.15.155:0/2910809794 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0814003aa0 con 0x7f0824107660 2024-09-19T06:11:40.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.209+0000 7f08217fa640 1 -- 172.21.15.155:0/2910809794 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f0814035030 con 0x7f0824107660 2024-09-19T06:11:40.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.209+0000 7f082a1ca640 1 -- 172.21.15.155:0/2910809794 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f082419b8f0 con 0x7f0824107660 2024-09-19T06:11:40.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.210+0000 7f08217fa640 1 -- 172.21.15.155:0/2910809794 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 46589+0+0 (secure 0 0 0) 0x7f0814042070 con 0x7f0824107660 2024-09-19T06:11:40.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.210+0000 7f082a1ca640 1 -- 172.21.15.155:0/2910809794 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0824102cd0 con 0x7f0824107660 2024-09-19T06:11:40.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.210+0000 7f08217fa640 1 --2- 172.21.15.155:0/2910809794 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f08100393d0 0x7f081003b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:40.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.210+0000 7f08217fa640 1 -- 172.21.15.155:0/2910809794 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f081406d2a0 con 0x7f0824107660 2024-09-19T06:11:40.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.210+0000 7f081affd640 1 --2- 172.21.15.155:0/2910809794 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f08100393d0 0x7f081003b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:40.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.211+0000 7f081affd640 1 --2- 172.21.15.155:0/2910809794 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f08100393d0 0x7f081003b890 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f080c0098c0 tx=0x7f080c006db0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:40.217 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.215+0000 7f08217fa640 1 -- 172.21.15.155:0/2910809794 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0814036260 con 0x7f0824107660 2024-09-19T06:11:40.450 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.448+0000 7f082a1ca640 1 -- 172.21.15.155:0/2910809794 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f0824198380 con 0x7f0824107660 2024-09-19T06:11:40.451 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.449+0000 7f08217fa640 1 -- 172.21.15.155:0/2910809794 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+753 (secure 0 0 0) 0x7f0814040d50 con 0x7f0824107660 2024-09-19T06:11:40.451 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:11:40.451 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":1,"fsid":"9f2876d0-764d-11ef-bceb-c7b262605968","modified":"2024-09-19T06:09:23.791881Z","created":"2024-09-19T06:09:23.791881Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi053","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T06:11:40.451 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 1 2024-09-19T06:11:40.453 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.451+0000 7f082a1ca640 1 -- 172.21.15.155:0/2910809794 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f08100393d0 msgr2=0x7f081003b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:40.453 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.451+0000 7f082a1ca640 1 --2- 172.21.15.155:0/2910809794 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f08100393d0 0x7f081003b890 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f080c0098c0 tx=0x7f080c006db0 comp rx=0 tx=0).stop 2024-09-19T06:11:40.453 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.451+0000 7f082a1ca640 1 -- 172.21.15.155:0/2910809794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0824107660 msgr2=0x7f0824197020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:40.454 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.451+0000 7f082a1ca640 1 --2- 172.21.15.155:0/2910809794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0824107660 0x7f0824197020 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0814006010 tx=0x7f08140048f0 comp rx=0 tx=0).stop 2024-09-19T06:11:40.454 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.451+0000 7f082a1ca640 1 -- 172.21.15.155:0/2910809794 shutdown_connections 2024-09-19T06:11:40.454 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.451+0000 7f082a1ca640 1 --2- 172.21.15.155:0/2910809794 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f08100393d0 0x7f081003b890 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:40.454 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.451+0000 7f082a1ca640 1 --2- 172.21.15.155:0/2910809794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0824107660 0x7f0824197020 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:40.454 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.451+0000 7f082a1ca640 1 -- 172.21.15.155:0/2910809794 >> 172.21.15.155:0/2910809794 conn(0x7f08240fd290 msgr2=0x7f08240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:40.454 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.451+0000 7f082a1ca640 1 -- 172.21.15.155:0/2910809794 shutdown_connections 2024-09-19T06:11:40.454 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:40.451+0000 7f082a1ca640 1 -- 172.21.15.155:0/2910809794 wait complete. 2024-09-19T06:11:41.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:40 smithi053 ceph-mon[27968]: from='client.? 172.21.15.155:0/2910809794' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:42.362 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-19T06:11:42.362 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T06:11:42.736 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:11:43.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.172+0000 7f8153682640 1 -- 172.21.15.155:0/98612700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f814c107660 msgr2=0x7f814c107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:43.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.172+0000 7f8153682640 1 --2- 172.21.15.155:0/98612700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f814c107660 0x7f814c107a40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8148009880 tx=0x7f814802a5d0 comp rx=0 tx=0).stop 2024-09-19T06:11:43.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.172+0000 7f8153682640 1 -- 172.21.15.155:0/98612700 shutdown_connections 2024-09-19T06:11:43.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.172+0000 7f8153682640 1 --2- 172.21.15.155:0/98612700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f814c107660 0x7f814c107a40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:43.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.172+0000 7f8153682640 1 -- 172.21.15.155:0/98612700 >> 172.21.15.155:0/98612700 conn(0x7f814c0fd290 msgr2=0x7f814c0ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:43.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.172+0000 7f8153682640 1 -- 172.21.15.155:0/98612700 shutdown_connections 2024-09-19T06:11:43.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.172+0000 7f8153682640 1 -- 172.21.15.155:0/98612700 wait complete. 2024-09-19T06:11:43.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.173+0000 7f8153682640 1 Processor -- start 2024-09-19T06:11:43.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.173+0000 7f8153682640 1 -- start start 2024-09-19T06:11:43.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.173+0000 7f8153682640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f814c107660 0x7f814c197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:43.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.173+0000 7f8153682640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f814c10bb40 con 0x7f814c107660 2024-09-19T06:11:43.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.173+0000 7f815141e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f814c107660 0x7f814c197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:43.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.173+0000 7f815141e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f814c107660 0x7f814c197020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.155:58444/0 (socket says 172.21.15.155:58444) 2024-09-19T06:11:43.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.174+0000 7f815141e640 1 -- 172.21.15.155:0/1037565319 learned_addr learned my addr 172.21.15.155:0/1037565319 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-19T06:11:43.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.174+0000 7f815141e640 1 -- 172.21.15.155:0/1037565319 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f814c197560 con 0x7f814c107660 2024-09-19T06:11:43.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.174+0000 7f815141e640 1 --2- 172.21.15.155:0/1037565319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f814c107660 0x7f814c197020 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8148006010 tx=0x7f81480048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:43.178 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.174+0000 7f813e7fc640 1 -- 172.21.15.155:0/1037565319 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f814803f070 con 0x7f814c107660 2024-09-19T06:11:43.178 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.174+0000 7f8153682640 1 -- 172.21.15.155:0/1037565319 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f814c1977f0 con 0x7f814c107660 2024-09-19T06:11:43.178 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.174+0000 7f813e7fc640 1 -- 172.21.15.155:0/1037565319 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8148003aa0 con 0x7f814c107660 2024-09-19T06:11:43.178 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.174+0000 7f813e7fc640 1 -- 172.21.15.155:0/1037565319 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f8148035030 con 0x7f814c107660 2024-09-19T06:11:43.178 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.174+0000 7f8153682640 1 -- 172.21.15.155:0/1037565319 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f814c19b8f0 con 0x7f814c107660 2024-09-19T06:11:43.178 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.175+0000 7f813e7fc640 1 -- 172.21.15.155:0/1037565319 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 46589+0+0 (secure 0 0 0) 0x7f8148042070 con 0x7f814c107660 2024-09-19T06:11:43.178 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.175+0000 7f8153682640 1 -- 172.21.15.155:0/1037565319 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f814c102cd0 con 0x7f814c107660 2024-09-19T06:11:43.178 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.175+0000 7f813e7fc640 1 --2- 172.21.15.155:0/1037565319 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f81380393d0 0x7f813803b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:43.178 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.176+0000 7f813e7fc640 1 -- 172.21.15.155:0/1037565319 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f814806d2e0 con 0x7f814c107660 2024-09-19T06:11:43.178 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.176+0000 7f8150c1d640 1 --2- 172.21.15.155:0/1037565319 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f81380393d0 0x7f813803b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:43.178 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.176+0000 7f8150c1d640 1 --2- 172.21.15.155:0/1037565319 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f81380393d0 0x7f813803b890 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f81400098c0 tx=0x7f8140006db0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:43.182 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.181+0000 7f813e7fc640 1 -- 172.21.15.155:0/1037565319 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8148036260 con 0x7f814c107660 2024-09-19T06:11:43.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:42 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:43.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:42 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:43.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:42 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:43.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:42 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:43.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:42 smithi053 ceph-mon[27968]: Deploying daemon node-exporter.smithi155 on smithi155 2024-09-19T06:11:43.421 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.420+0000 7f8153682640 1 -- 172.21.15.155:0/1037565319 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f814c198380 con 0x7f814c107660 2024-09-19T06:11:43.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.421+0000 7f813e7fc640 1 -- 172.21.15.155:0/1037565319 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+753 (secure 0 0 0) 0x7f8148040d50 con 0x7f814c107660 2024-09-19T06:11:43.422 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:11:43.422 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":1,"fsid":"9f2876d0-764d-11ef-bceb-c7b262605968","modified":"2024-09-19T06:09:23.791881Z","created":"2024-09-19T06:09:23.791881Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi053","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T06:11:43.423 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 1 2024-09-19T06:11:43.424 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.423+0000 7f8153682640 1 -- 172.21.15.155:0/1037565319 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f81380393d0 msgr2=0x7f813803b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:43.424 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.423+0000 7f8153682640 1 --2- 172.21.15.155:0/1037565319 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f81380393d0 0x7f813803b890 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f81400098c0 tx=0x7f8140006db0 comp rx=0 tx=0).stop 2024-09-19T06:11:43.424 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.423+0000 7f8153682640 1 -- 172.21.15.155:0/1037565319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f814c107660 msgr2=0x7f814c197020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:43.425 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.423+0000 7f8153682640 1 --2- 172.21.15.155:0/1037565319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f814c107660 0x7f814c197020 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8148006010 tx=0x7f81480048f0 comp rx=0 tx=0).stop 2024-09-19T06:11:43.425 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.423+0000 7f8153682640 1 -- 172.21.15.155:0/1037565319 shutdown_connections 2024-09-19T06:11:43.425 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.423+0000 7f8153682640 1 --2- 172.21.15.155:0/1037565319 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f81380393d0 0x7f813803b890 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:43.425 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.423+0000 7f8153682640 1 --2- 172.21.15.155:0/1037565319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f814c107660 0x7f814c197020 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:43.425 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.423+0000 7f8153682640 1 -- 172.21.15.155:0/1037565319 >> 172.21.15.155:0/1037565319 conn(0x7f814c0fd290 msgr2=0x7f814c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:43.425 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.423+0000 7f8153682640 1 -- 172.21.15.155:0/1037565319 shutdown_connections 2024-09-19T06:11:43.425 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:43.423+0000 7f8153682640 1 -- 172.21.15.155:0/1037565319 wait complete. 2024-09-19T06:11:44.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:43 smithi053 ceph-mon[27968]: from='client.? 172.21.15.155:0/1037565319' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:44.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:43 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:44.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:43 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:11:45.251 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-19T06:11:45.251 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T06:11:45.708 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:11:46.347 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.333+0000 7f2a0d82a640 1 -- 172.21.15.155:0/1645788903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a08102750 msgr2=0x7f2a08102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:46.347 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.333+0000 7f2a0d82a640 1 --2- 172.21.15.155:0/1645788903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a08102750 0x7f2a08102b30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f29f8009880 tx=0x7f29f802a670 comp rx=0 tx=0).stop 2024-09-19T06:11:46.347 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.333+0000 7f2a0d82a640 1 -- 172.21.15.155:0/1645788903 shutdown_connections 2024-09-19T06:11:46.347 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.333+0000 7f2a0d82a640 1 --2- 172.21.15.155:0/1645788903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a08102750 0x7f2a08102b30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:46.347 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.333+0000 7f2a0d82a640 1 -- 172.21.15.155:0/1645788903 >> 172.21.15.155:0/1645788903 conn(0x7f2a080fc1b0 msgr2=0x7f2a080fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:46.347 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.333+0000 7f2a0d82a640 1 -- 172.21.15.155:0/1645788903 shutdown_connections 2024-09-19T06:11:46.347 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.333+0000 7f2a0d82a640 1 -- 172.21.15.155:0/1645788903 wait complete. 2024-09-19T06:11:46.347 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.334+0000 7f2a0d82a640 1 Processor -- start 2024-09-19T06:11:46.347 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.334+0000 7f2a0d82a640 1 -- start start 2024-09-19T06:11:46.347 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.334+0000 7f2a0d82a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a08102750 0x7f2a081003b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:46.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.334+0000 7f2a0d82a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0810bb70 con 0x7f2a08102750 2024-09-19T06:11:46.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.335+0000 7f2a06ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a08102750 0x7f2a081003b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:46.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.335+0000 7f2a06ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a08102750 0x7f2a081003b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.155:58460/0 (socket says 172.21.15.155:58460) 2024-09-19T06:11:46.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.335+0000 7f2a06ffd640 1 -- 172.21.15.155:0/3780155486 learned_addr learned my addr 172.21.15.155:0/3780155486 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-19T06:11:46.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.335+0000 7f2a06ffd640 1 -- 172.21.15.155:0/3780155486 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a081022a0 con 0x7f2a08102750 2024-09-19T06:11:46.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.335+0000 7f2a06ffd640 1 --2- 172.21.15.155:0/3780155486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a08102750 0x7f2a081003b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f29f8006010 tx=0x7f29f80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:46.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.335+0000 7f2a0c828640 1 -- 172.21.15.155:0/3780155486 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f29f803f070 con 0x7f2a08102750 2024-09-19T06:11:46.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.335+0000 7f2a0d82a640 1 -- 172.21.15.155:0/3780155486 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a081008f0 con 0x7f2a08102750 2024-09-19T06:11:46.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.335+0000 7f2a0c828640 1 -- 172.21.15.155:0/3780155486 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29f8003aa0 con 0x7f2a08102750 2024-09-19T06:11:46.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.336+0000 7f2a0d82a640 1 -- 172.21.15.155:0/3780155486 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a08100db0 con 0x7f2a08102750 2024-09-19T06:11:46.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.336+0000 7f2a0c828640 1 -- 172.21.15.155:0/3780155486 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f29f8049060 con 0x7f2a08102750 2024-09-19T06:11:46.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.336+0000 7f2a0c828640 1 -- 172.21.15.155:0/3780155486 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 46589+0+0 (secure 0 0 0) 0x7f29f8042070 con 0x7f2a08102750 2024-09-19T06:11:46.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.337+0000 7f2a0d82a640 1 -- 172.21.15.155:0/3780155486 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a0806a7b0 con 0x7f2a08102750 2024-09-19T06:11:46.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.337+0000 7f2a0c828640 1 --2- 172.21.15.155:0/3780155486 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f29f40393d0 0x7f29f403b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:46.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.337+0000 7f2a0c828640 1 -- 172.21.15.155:0/3780155486 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f29f806d190 con 0x7f2a08102750 2024-09-19T06:11:46.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.337+0000 7f29fffff640 1 --2- 172.21.15.155:0/3780155486 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f29f40393d0 0x7f29f403b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:46.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.337+0000 7f29fffff640 1 --2- 172.21.15.155:0/3780155486 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f29f40393d0 0x7f29f403b890 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f29f00098c0 tx=0x7f29f0006db0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:46.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.342+0000 7f2a0c828640 1 -- 172.21.15.155:0/3780155486 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f29f8040260 con 0x7f2a08102750 2024-09-19T06:11:46.577 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.576+0000 7f2a0d82a640 1 -- 172.21.15.155:0/3780155486 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f2a081018e0 con 0x7f2a08102750 2024-09-19T06:11:46.578 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.576+0000 7f2a0c828640 1 -- 172.21.15.155:0/3780155486 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+753 (secure 0 0 0) 0x7f29f803ade0 con 0x7f2a08102750 2024-09-19T06:11:46.578 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:11:46.578 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":1,"fsid":"9f2876d0-764d-11ef-bceb-c7b262605968","modified":"2024-09-19T06:09:23.791881Z","created":"2024-09-19T06:09:23.791881Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi053","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T06:11:46.579 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 1 2024-09-19T06:11:46.580 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.579+0000 7f2a0d82a640 1 -- 172.21.15.155:0/3780155486 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f29f40393d0 msgr2=0x7f29f403b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:46.580 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.579+0000 7f2a0d82a640 1 --2- 172.21.15.155:0/3780155486 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f29f40393d0 0x7f29f403b890 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f29f00098c0 tx=0x7f29f0006db0 comp rx=0 tx=0).stop 2024-09-19T06:11:46.581 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.579+0000 7f2a0d82a640 1 -- 172.21.15.155:0/3780155486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a08102750 msgr2=0x7f2a081003b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:46.581 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.579+0000 7f2a0d82a640 1 --2- 172.21.15.155:0/3780155486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a08102750 0x7f2a081003b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f29f8006010 tx=0x7f29f80048f0 comp rx=0 tx=0).stop 2024-09-19T06:11:46.581 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.579+0000 7f2a0d82a640 1 -- 172.21.15.155:0/3780155486 shutdown_connections 2024-09-19T06:11:46.581 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.579+0000 7f2a0d82a640 1 --2- 172.21.15.155:0/3780155486 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f29f40393d0 0x7f29f403b890 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:46.581 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.579+0000 7f2a0d82a640 1 --2- 172.21.15.155:0/3780155486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a08102750 0x7f2a081003b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:46.581 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.579+0000 7f2a0d82a640 1 -- 172.21.15.155:0/3780155486 >> 172.21.15.155:0/3780155486 conn(0x7f2a080fc1b0 msgr2=0x7f2a08105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:46.581 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.579+0000 7f2a0d82a640 1 -- 172.21.15.155:0/3780155486 shutdown_connections 2024-09-19T06:11:46.581 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:46.579+0000 7f2a0d82a640 1 -- 172.21.15.155:0/3780155486 wait complete. 2024-09-19T06:11:47.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:46 smithi053 ceph-mon[27968]: from='client.? 172.21.15.155:0/3780155486' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:48.305 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-19T06:11:48.305 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T06:11:48.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:47 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:48.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:47 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:48.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:47 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:48.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:47 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:48.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:47 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi155.piymgv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T06:11:48.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:47 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi155.piymgv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T06:11:48.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:47 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T06:11:48.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:47 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:11:48.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:47 smithi053 ceph-mon[27968]: Deploying daemon mgr.smithi155.piymgv on smithi155 2024-09-19T06:11:48.567 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:11:49.101 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.083+0000 7fadbc3ee640 1 -- 172.21.15.155:0/2654879816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadb4106560 msgr2=0x7fadb4106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:49.101 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.083+0000 7fadbc3ee640 1 --2- 172.21.15.155:0/2654879816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadb4106560 0x7fadb4106940 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fada8009880 tx=0x7fada802a5d0 comp rx=0 tx=0).stop 2024-09-19T06:11:49.102 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.083+0000 7fadbc3ee640 1 -- 172.21.15.155:0/2654879816 shutdown_connections 2024-09-19T06:11:49.102 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.083+0000 7fadbc3ee640 1 --2- 172.21.15.155:0/2654879816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadb4106560 0x7fadb4106940 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:49.102 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.083+0000 7fadbc3ee640 1 -- 172.21.15.155:0/2654879816 >> 172.21.15.155:0/2654879816 conn(0x7fadb40fc1b0 msgr2=0x7fadb40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:49.102 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.083+0000 7fadbc3ee640 1 -- 172.21.15.155:0/2654879816 shutdown_connections 2024-09-19T06:11:49.102 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.083+0000 7fadbc3ee640 1 -- 172.21.15.155:0/2654879816 wait complete. 2024-09-19T06:11:49.102 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.083+0000 7fadbc3ee640 1 Processor -- start 2024-09-19T06:11:49.102 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.084+0000 7fadbc3ee640 1 -- start start 2024-09-19T06:11:49.102 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.084+0000 7fadbc3ee640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadb4106560 0x7fadb4197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:49.102 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.084+0000 7fadbc3ee640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fadb410aa40 con 0x7fadb4106560 2024-09-19T06:11:49.102 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.084+0000 7fadba18a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadb4106560 0x7fadb4197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:49.102 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.084+0000 7fadba18a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadb4106560 0x7fadb4197040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.155:58476/0 (socket says 172.21.15.155:58476) 2024-09-19T06:11:49.102 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.084+0000 7fadba18a640 1 -- 172.21.15.155:0/909868857 learned_addr learned my addr 172.21.15.155:0/909868857 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-19T06:11:49.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.084+0000 7fadba18a640 1 -- 172.21.15.155:0/909868857 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb4197580 con 0x7fadb4106560 2024-09-19T06:11:49.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.085+0000 7fadba18a640 1 --2- 172.21.15.155:0/909868857 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadb4106560 0x7fadb4197040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fada8006010 tx=0x7fada80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:49.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.085+0000 7fada77fe640 1 -- 172.21.15.155:0/909868857 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fada803f070 con 0x7fadb4106560 2024-09-19T06:11:49.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.085+0000 7fadbc3ee640 1 -- 172.21.15.155:0/909868857 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadb4197810 con 0x7fadb4106560 2024-09-19T06:11:49.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.085+0000 7fada77fe640 1 -- 172.21.15.155:0/909868857 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fada8003aa0 con 0x7fadb4106560 2024-09-19T06:11:49.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.085+0000 7fada77fe640 1 -- 172.21.15.155:0/909868857 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7fada8035030 con 0x7fadb4106560 2024-09-19T06:11:49.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.085+0000 7fadbc3ee640 1 -- 172.21.15.155:0/909868857 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadb419b910 con 0x7fadb4106560 2024-09-19T06:11:49.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.086+0000 7fadbc3ee640 1 -- 172.21.15.155:0/909868857 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadb4101bd0 con 0x7fadb4106560 2024-09-19T06:11:49.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.086+0000 7fada77fe640 1 -- 172.21.15.155:0/909868857 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 46589+0+0 (secure 0 0 0) 0x7fada8042070 con 0x7fadb4106560 2024-09-19T06:11:49.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.086+0000 7fada77fe640 1 --2- 172.21.15.155:0/909868857 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fada00393d0 0x7fada003b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:49.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.086+0000 7fada77fe640 1 -- 172.21.15.155:0/909868857 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fada806d220 con 0x7fadb4106560 2024-09-19T06:11:49.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.086+0000 7fadb9989640 1 --2- 172.21.15.155:0/909868857 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fada00393d0 0x7fada003b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:49.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.087+0000 7fadb9989640 1 --2- 172.21.15.155:0/909868857 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fada00393d0 0x7fada003b890 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fadb00098c0 tx=0x7fadb0006db0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:49.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.091+0000 7fada77fe640 1 -- 172.21.15.155:0/909868857 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fada8036260 con 0x7fadb4106560 2024-09-19T06:11:49.334 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.332+0000 7fadbc3ee640 1 -- 172.21.15.155:0/909868857 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fadb41983a0 con 0x7fadb4106560 2024-09-19T06:11:49.334 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.333+0000 7fada77fe640 1 -- 172.21.15.155:0/909868857 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+753 (secure 0 0 0) 0x7fada8040d50 con 0x7fadb4106560 2024-09-19T06:11:49.335 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:11:49.335 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":1,"fsid":"9f2876d0-764d-11ef-bceb-c7b262605968","modified":"2024-09-19T06:09:23.791881Z","created":"2024-09-19T06:09:23.791881Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi053","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T06:11:49.335 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 1 2024-09-19T06:11:49.337 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.335+0000 7fadbc3ee640 1 -- 172.21.15.155:0/909868857 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fada00393d0 msgr2=0x7fada003b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:49.337 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.335+0000 7fadbc3ee640 1 --2- 172.21.15.155:0/909868857 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fada00393d0 0x7fada003b890 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fadb00098c0 tx=0x7fadb0006db0 comp rx=0 tx=0).stop 2024-09-19T06:11:49.337 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.335+0000 7fadbc3ee640 1 -- 172.21.15.155:0/909868857 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadb4106560 msgr2=0x7fadb4197040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:49.337 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.335+0000 7fadbc3ee640 1 --2- 172.21.15.155:0/909868857 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadb4106560 0x7fadb4197040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fada8006010 tx=0x7fada80048f0 comp rx=0 tx=0).stop 2024-09-19T06:11:49.337 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.335+0000 7fadbc3ee640 1 -- 172.21.15.155:0/909868857 shutdown_connections 2024-09-19T06:11:49.337 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.335+0000 7fadbc3ee640 1 --2- 172.21.15.155:0/909868857 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fada00393d0 0x7fada003b890 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:49.337 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.335+0000 7fadbc3ee640 1 --2- 172.21.15.155:0/909868857 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadb4106560 0x7fadb4197040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:49.337 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.335+0000 7fadbc3ee640 1 -- 172.21.15.155:0/909868857 >> 172.21.15.155:0/909868857 conn(0x7fadb40fc1b0 msgr2=0x7fadb4069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:49.337 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.335+0000 7fadbc3ee640 1 -- 172.21.15.155:0/909868857 shutdown_connections 2024-09-19T06:11:49.338 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:49.335+0000 7fadbc3ee640 1 -- 172.21.15.155:0/909868857 wait complete. 2024-09-19T06:11:49.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:48 smithi053 ceph-mon[27968]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:49.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:48 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:50.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:49 smithi053 ceph-mon[27968]: from='client.? 172.21.15.155:0/909868857' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:51.245 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-19T06:11:51.246 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T06:11:51.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:51 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:51.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:51 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:51.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:51 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:51.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:51 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:51.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:51 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T06:11:51.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:51 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:11:51.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:51 smithi053 ceph-mon[27968]: Deploying daemon mon.smithi155 on smithi155 2024-09-19T06:11:51.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:51 smithi053 ceph-mon[27968]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:52.529 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi155/config 2024-09-19T06:11:53.078 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.044+0000 7f123510e640 1 -- 172.21.15.155:0/1464240485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1230102870 msgr2=0x7f1230102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:53.078 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.044+0000 7f123510e640 1 --2- 172.21.15.155:0/1464240485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1230102870 0x7f1230102c50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1220009c20 tx=0x7f122002ab20 comp rx=0 tx=0).stop 2024-09-19T06:11:53.078 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.044+0000 7f123510e640 1 -- 172.21.15.155:0/1464240485 shutdown_connections 2024-09-19T06:11:53.078 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.044+0000 7f123510e640 1 --2- 172.21.15.155:0/1464240485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1230102870 0x7f1230102c50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:53.078 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.044+0000 7f123510e640 1 -- 172.21.15.155:0/1464240485 >> 172.21.15.155:0/1464240485 conn(0x7f12300fc290 msgr2=0x7f12300fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:53.078 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.044+0000 7f123510e640 1 -- 172.21.15.155:0/1464240485 shutdown_connections 2024-09-19T06:11:53.079 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.044+0000 7f123510e640 1 -- 172.21.15.155:0/1464240485 wait complete. 2024-09-19T06:11:53.079 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.046+0000 7f123510e640 1 Processor -- start 2024-09-19T06:11:53.079 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.046+0000 7f123510e640 1 -- start start 2024-09-19T06:11:53.079 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.046+0000 7f123510e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1230102870 0x7f12301993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:53.079 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.046+0000 7f123510e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f123010bc90 con 0x7f1230102870 2024-09-19T06:11:53.079 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.047+0000 7f122ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1230102870 0x7f12301993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:53.079 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.047+0000 7f122ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1230102870 0x7f12301993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.155:38360/0 (socket says 172.21.15.155:38360) 2024-09-19T06:11:53.079 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.047+0000 7f122ed9d640 1 -- 172.21.15.155:0/1993350318 learned_addr learned my addr 172.21.15.155:0/1993350318 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-19T06:11:53.079 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.047+0000 7f122ed9d640 1 -- 172.21.15.155:0/1993350318 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1230199910 con 0x7f1230102870 2024-09-19T06:11:53.079 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.047+0000 7f122ed9d640 1 --2- 172.21.15.155:0/1993350318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1230102870 0x7f12301993d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f122002eb50 tx=0x7f122002eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:53.079 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.047+0000 7f1227fff640 1 -- 172.21.15.155:0/1993350318 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f1220047070 con 0x7f1230102870 2024-09-19T06:11:53.079 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.047+0000 7f1227fff640 1 -- 172.21.15.155:0/1993350318 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12200368b0 con 0x7f1230102870 2024-09-19T06:11:53.079 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.047+0000 7f123510e640 1 -- 172.21.15.155:0/1993350318 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1230199ba0 con 0x7f1230102870 2024-09-19T06:11:53.079 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.047+0000 7f1227fff640 1 -- 172.21.15.155:0/1993350318 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 229+0+0 (secure 0 0 0) 0x7f1220004840 con 0x7f1230102870 2024-09-19T06:11:53.080 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.048+0000 7f123510e640 1 -- 172.21.15.155:0/1993350318 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f123019a000 con 0x7f1230102870 2024-09-19T06:11:53.080 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.048+0000 7f1227fff640 1 -- 172.21.15.155:0/1993350318 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 46589+0+0 (secure 0 0 0) 0x7f122004a070 con 0x7f1230102870 2024-09-19T06:11:53.080 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.049+0000 7f1227fff640 1 --2- 172.21.15.155:0/1993350318 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f121c039370 0x7f121c03b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:53.080 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.049+0000 7f1227fff640 1 -- 172.21.15.155:0/1993350318 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f122003d030 con 0x7f1230102870 2024-09-19T06:11:53.080 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.049+0000 7f123510e640 1 -- 172.21.15.155:0/1993350318 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f123006ad10 con 0x7f1230102870 2024-09-19T06:11:53.080 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.049+0000 7f122e59c640 1 --2- 172.21.15.155:0/1993350318 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f121c039370 0x7f121c03b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:53.080 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.049+0000 7f122e59c640 1 --2- 172.21.15.155:0/1993350318 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f121c039370 0x7f121c03b830 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f12180098c0 tx=0x7f1218006db0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:53.080 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.054+0000 7f1227fff640 1 -- 172.21.15.155:0/1993350318 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1220051080 con 0x7f1230102870 2024-09-19T06:11:53.080 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 systemd[1]: Starting Ceph mon.smithi155 for 9f2876d0-764d-11ef-bceb-c7b262605968... 2024-09-19T06:11:53.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:52 smithi053 ceph-mon[27968]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:53.321 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.319+0000 7f123510e640 1 -- 172.21.15.155:0/1993350318 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f123019a730 con 0x7f1230102870 2024-09-19T06:11:53.322 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.320+0000 7f1227fff640 1 -- 172.21.15.155:0/1993350318 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+753 (secure 0 0 0) 0x7f1220032030 con 0x7f1230102870 2024-09-19T06:11:53.322 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 1 2024-09-19T06:11:53.322 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:11:53.322 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":1,"fsid":"9f2876d0-764d-11ef-bceb-c7b262605968","modified":"2024-09-19T06:09:23.791881Z","created":"2024-09-19T06:09:23.791881Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi053","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T06:11:53.324 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.322+0000 7f123510e640 1 -- 172.21.15.155:0/1993350318 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f121c039370 msgr2=0x7f121c03b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:53.324 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.322+0000 7f123510e640 1 --2- 172.21.15.155:0/1993350318 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f121c039370 0x7f121c03b830 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f12180098c0 tx=0x7f1218006db0 comp rx=0 tx=0).stop 2024-09-19T06:11:53.324 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.322+0000 7f123510e640 1 -- 172.21.15.155:0/1993350318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1230102870 msgr2=0x7f12301993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:53.324 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.322+0000 7f123510e640 1 --2- 172.21.15.155:0/1993350318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1230102870 0x7f12301993d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f122002eb50 tx=0x7f122002eb80 comp rx=0 tx=0).stop 2024-09-19T06:11:53.324 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.322+0000 7f123510e640 1 -- 172.21.15.155:0/1993350318 shutdown_connections 2024-09-19T06:11:53.324 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.322+0000 7f123510e640 1 --2- 172.21.15.155:0/1993350318 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f121c039370 0x7f121c03b830 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:53.324 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.322+0000 7f123510e640 1 --2- 172.21.15.155:0/1993350318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1230102870 0x7f12301993d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:53.324 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.322+0000 7f123510e640 1 -- 172.21.15.155:0/1993350318 >> 172.21.15.155:0/1993350318 conn(0x7f12300fc290 msgr2=0x7f1230107a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:53.324 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.322+0000 7f123510e640 1 -- 172.21.15.155:0/1993350318 shutdown_connections 2024-09-19T06:11:53.325 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:53.322+0000 7f123510e640 1 -- 172.21.15.155:0/1993350318 wait complete. 2024-09-19T06:11:53.333 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 podman[35261]: 2024-09-19 06:11:53.187643742 +0000 UTC m=+0.020649299 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:11:53.334 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 podman[35261]: 2024-09-19 06:11:53.294331393 +0000 UTC m=+0.127336948 container create 043f44f8feb0f1dd9f903a987dd746c2ffce335378a7bf6f1f27939b6d1bc8eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-mon-smithi155, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-19T06:11:53.929 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 podman[35261]: 2024-09-19 06:11:53.781642022 +0000 UTC m=+0.614647590 container init 043f44f8feb0f1dd9f903a987dd746c2ffce335378a7bf6f1f27939b6d1bc8eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-mon-smithi155, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:11:53.929 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 podman[35261]: 2024-09-19 06:11:53.785444932 +0000 UTC m=+0.618450492 container start 043f44f8feb0f1dd9f903a987dd746c2ffce335378a7bf6f1f27939b6d1bc8eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-mon-smithi155, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-19T06:11:53.929 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T06:11:53.929 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T06:11:53.929 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: pidfile_write: ignore empty --pid-file 2024-09-19T06:11:53.929 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: load: jerasure load: lrc 2024-09-19T06:11:53.930 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: RocksDB version: 7.9.2 2024-09-19T06:11:53.930 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Git sha 0 2024-09-19T06:11:53.930 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T06:11:53.930 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: DB SUMMARY 2024-09-19T06:11:53.930 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: DB Session ID: YEYN3GUYM957FWRWQ75B 2024-09-19T06:11:53.930 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: CURRENT file: CURRENT 2024-09-19T06:11:53.930 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T06:11:53.930 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-19T06:11:53.930 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi155/store.db dir, Total Num: 0, files: 2024-09-19T06:11:53.930 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi155/store.db: 000004.log size: 511 ; 2024-09-19T06:11:53.930 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.error_if_exists: 0 2024-09-19T06:11:53.930 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.create_if_missing: 0 2024-09-19T06:11:53.930 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.paranoid_checks: 1 2024-09-19T06:11:53.930 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T06:11:53.931 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T06:11:53.931 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T06:11:53.931 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.env: 0x5607aaa0d940 2024-09-19T06:11:53.931 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T06:11:53.931 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.info_log: 0x5607ac6458c0 2024-09-19T06:11:53.931 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T06:11:53.933 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.statistics: (nil) 2024-09-19T06:11:53.933 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.use_fsync: 0 2024-09-19T06:11:53.933 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.max_log_file_size: 0 2024-09-19T06:11:53.934 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T06:11:53.934 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T06:11:53.934 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T06:11:53.934 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T06:11:53.934 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.allow_fallocate: 1 2024-09-19T06:11:53.934 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T06:11:53.934 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T06:11:53.934 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.use_direct_reads: 0 2024-09-19T06:11:53.934 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T06:11:53.934 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T06:11:53.934 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.db_log_dir: 2024-09-19T06:11:53.934 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.wal_dir: 2024-09-19T06:11:53.934 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T06:11:53.934 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T06:11:53.935 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T06:11:53.935 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T06:11:53.935 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T06:11:53.935 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T06:11:53.935 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T06:11:53.935 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T06:11:53.935 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.write_buffer_manager: 0x5607ac654dc0 2024-09-19T06:11:53.935 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T06:11:53.935 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T06:11:53.935 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T06:11:53.935 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T06:11:53.935 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T06:11:53.935 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T06:11:53.936 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T06:11:53.936 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T06:11:53.936 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.unordered_write: 0 2024-09-19T06:11:53.936 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T06:11:53.936 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T06:11:53.936 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T06:11:53.936 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T06:11:53.936 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.row_cache: None 2024-09-19T06:11:53.936 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.wal_filter: None 2024-09-19T06:11:53.936 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T06:11:53.936 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T06:11:53.936 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.two_write_queues: 0 2024-09-19T06:11:53.936 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T06:11:53.936 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.wal_compression: 0 2024-09-19T06:11:53.937 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.atomic_flush: 0 2024-09-19T06:11:53.937 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T06:11:53.937 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T06:11:53.937 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T06:11:53.937 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.log_readahead_size: 0 2024-09-19T06:11:53.937 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T06:11:53.937 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T06:11:53.937 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T06:11:53.937 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T06:11:53.937 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T06:11:53.937 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T06:11:53.937 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T06:11:53.937 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.max_background_jobs: 2 2024-09-19T06:11:53.937 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.max_background_compactions: -1 2024-09-19T06:11:53.938 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.max_subcompactions: 1 2024-09-19T06:11:53.938 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T06:11:53.938 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T06:11:53.938 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T06:11:53.938 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T06:11:53.938 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T06:11:53.938 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T06:11:53.938 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T06:11:53.938 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T06:11:53.938 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.max_open_files: -1 2024-09-19T06:11:53.938 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T06:11:53.938 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T06:11:53.938 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T06:11:53.938 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T06:11:53.938 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.max_background_flushes: -1 2024-09-19T06:11:53.939 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Compression algorithms supported: 2024-09-19T06:11:53.939 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: kZSTD supported: 0 2024-09-19T06:11:53.939 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: kXpressCompression supported: 0 2024-09-19T06:11:53.939 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: kBZip2Compression supported: 0 2024-09-19T06:11:53.939 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T06:11:53.939 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: kLZ4Compression supported: 1 2024-09-19T06:11:53.939 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: kZlibCompression supported: 1 2024-09-19T06:11:53.939 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T06:11:53.939 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: kSnappyCompression supported: 1 2024-09-19T06:11:53.939 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T06:11:53.939 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T06:11:53.939 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi155/store.db/MANIFEST-000005 2024-09-19T06:11:53.939 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T06:11:53.939 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T06:11:53.940 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.merge_operator: 2024-09-19T06:11:53.940 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.compaction_filter: None 2024-09-19T06:11:53.940 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T06:11:53.940 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T06:11:53.940 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T06:11:53.940 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T06:11:53.940 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5607ac645520) 2024-09-19T06:11:53.940 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: cache_index_and_filter_blocks: 1 2024-09-19T06:11:53.940 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T06:11:53.940 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T06:11:53.940 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: pin_top_level_index_and_filter: 1 2024-09-19T06:11:53.940 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: index_type: 0 2024-09-19T06:11:53.940 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: data_block_index_type: 0 2024-09-19T06:11:53.940 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: index_shortening: 1 2024-09-19T06:11:53.941 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T06:11:53.941 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: checksum: 4 2024-09-19T06:11:53.941 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: no_block_cache: 0 2024-09-19T06:11:53.941 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: block_cache: 0x5607ac6391f0 2024-09-19T06:11:53.941 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: block_cache_name: BinnedLRUCache 2024-09-19T06:11:53.941 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: block_cache_options: 2024-09-19T06:11:53.941 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: capacity : 536870912 2024-09-19T06:11:53.941 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: num_shard_bits : 4 2024-09-19T06:11:53.941 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: strict_capacity_limit : 0 2024-09-19T06:11:53.941 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: high_pri_pool_ratio: 0.000 2024-09-19T06:11:53.941 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: block_cache_compressed: (nil) 2024-09-19T06:11:53.941 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: persistent_cache: (nil) 2024-09-19T06:11:53.941 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: block_size: 4096 2024-09-19T06:11:53.942 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: block_size_deviation: 10 2024-09-19T06:11:53.942 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: block_restart_interval: 16 2024-09-19T06:11:53.942 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: index_block_restart_interval: 1 2024-09-19T06:11:53.942 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: metadata_block_size: 4096 2024-09-19T06:11:53.942 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: partition_filters: 0 2024-09-19T06:11:53.942 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: use_delta_encoding: 1 2024-09-19T06:11:53.942 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: filter_policy: bloomfilter 2024-09-19T06:11:53.942 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: whole_key_filtering: 1 2024-09-19T06:11:53.942 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: verify_compression: 0 2024-09-19T06:11:53.942 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: read_amp_bytes_per_bit: 0 2024-09-19T06:11:53.942 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: format_version: 5 2024-09-19T06:11:53.942 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: enable_index_compression: 1 2024-09-19T06:11:53.942 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: block_align: 0 2024-09-19T06:11:53.942 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: max_auto_readahead_size: 262144 2024-09-19T06:11:53.942 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: prepopulate_block_cache: 0 2024-09-19T06:11:53.943 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: initial_auto_readahead_size: 8192 2024-09-19T06:11:53.943 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T06:11:53.943 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T06:11:53.943 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T06:11:53.943 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.compression: NoCompression 2024-09-19T06:11:53.943 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T06:11:53.943 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T06:11:53.943 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T06:11:53.943 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.num_levels: 7 2024-09-19T06:11:53.943 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T06:11:53.943 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T06:11:53.943 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T06:11:53.943 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T06:11:53.943 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T06:11:53.943 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T06:11:53.944 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T06:11:53.944 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T06:11:53.944 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T06:11:53.944 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T06:11:53.944 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T06:11:53.944 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T06:11:53.944 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T06:11:53.944 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T06:11:53.944 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T06:11:53.944 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T06:11:53.944 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T06:11:53.944 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T06:11:53.944 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T06:11:53.944 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T06:11:53.945 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T06:11:53.945 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T06:11:53.945 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T06:11:53.945 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T06:11:53.945 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T06:11:53.945 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T06:11:53.945 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T06:11:53.945 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T06:11:53.945 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T06:11:53.945 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T06:11:53.945 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T06:11:53.945 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T06:11:53.945 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T06:11:53.945 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T06:11:53.946 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T06:11:53.946 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T06:11:53.946 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T06:11:53.946 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T06:11:53.946 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T06:11:53.946 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T06:11:53.946 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T06:11:53.946 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T06:11:53.946 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T06:11:53.946 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T06:11:53.946 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T06:11:53.946 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T06:11:53.946 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T06:11:53.946 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T06:11:53.947 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T06:11:53.947 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T06:11:53.947 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T06:11:53.947 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T06:11:53.947 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T06:11:53.947 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T06:11:53.947 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.inplace_update_support: 0 2024-09-19T06:11:53.947 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T06:11:53.947 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T06:11:53.947 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T06:11:53.947 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T06:11:53.947 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.bloom_locality: 0 2024-09-19T06:11:53.947 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.max_successive_merges: 0 2024-09-19T06:11:53.947 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T06:11:53.947 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T06:11:53.948 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T06:11:53.948 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T06:11:53.948 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.ttl: 2592000 2024-09-19T06:11:53.948 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T06:11:53.948 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T06:11:53.948 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T06:11:53.948 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.enable_blob_files: false 2024-09-19T06:11:53.948 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.min_blob_size: 0 2024-09-19T06:11:53.948 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T06:11:53.948 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T06:11:53.948 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T06:11:53.948 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T06:11:53.948 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T06:11:53.948 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T06:11:53.949 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T06:11:53.949 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T06:11:53.949 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi155/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-19T06:11:53.949 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-19T06:11:53.949 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 074a04ca-6572-45b5-9beb-3cee4b83b4d1 2024-09-19T06:11:53.949 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726726313812393, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-19T06:11:53.949 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-19T06:11:53.949 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726726313812823, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726726313, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "074a04ca-6572-45b5-9beb-3cee4b83b4d1", "db_session_id": "YEYN3GUYM957FWRWQ75B", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-19T06:11:53.949 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726726313812919, "job": 1, "event": "recovery_finished"} 2024-09-19T06:11:53.949 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-19T06:11:53.949 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi155/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T06:11:53.949 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5607ac666e00 2024-09-19T06:11:53.949 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: DB pointer 0x5607ac762000 2024-09-19T06:11:53.949 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T06:11:53.950 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T06:11:53.950 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: ** DB Stats ** 2024-09-19T06:11:53.950 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T06:11:53.950 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T06:11:53.950 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T06:11:53.950 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T06:11:53.950 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T06:11:53.950 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T06:11:53.950 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T06:11:53.950 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: 2024-09-19T06:11:53.950 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: ** Compaction Stats [default] ** 2024-09-19T06:11:53.950 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T06:11:53.950 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T06:11:53.950 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T06:11:53.950 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T06:11:53.951 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T06:11:53.951 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: 2024-09-19T06:11:53.951 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: ** Compaction Stats [default] ** 2024-09-19T06:11:53.951 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T06:11:53.951 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T06:11:53.951 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T06:11:53.951 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: 2024-09-19T06:11:53.951 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T06:11:53.951 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: 2024-09-19T06:11:53.951 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T06:11:53.951 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T06:11:53.951 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T06:11:53.951 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T06:11:53.951 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T06:11:53.952 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T06:11:53.952 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: Cumulative compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T06:11:53.952 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: Interval compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T06:11:53.952 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T06:11:53.952 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: Block cache BinnedLRUCache@0x5607ac6391f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-19T06:11:53.952 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-19T06:11:53.952 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: 2024-09-19T06:11:53.952 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T06:11:53.952 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: mon.smithi155 does not exist in monmap, will attempt to join an existing cluster 2024-09-19T06:11:53.952 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: using public_addr v2:172.21.15.155:0/0 -> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] 2024-09-19T06:11:53.952 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: starting mon.smithi155 rank -1 at public addrs [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] at bind addrs [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi155 fsid 9f2876d0-764d-11ef-bceb-c7b262605968 2024-09-19T06:11:53.952 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: mon.smithi155@-1(???) e0 preinit fsid 9f2876d0-764d-11ef-bceb-c7b262605968 2024-09-19T06:11:53.952 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: mon.smithi155@-1(synchronizing).mds e1 new map 2024-09-19T06:11:53.953 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: mon.smithi155@-1(synchronizing).mds e1 print_map 2024-09-19T06:11:53.953 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: e1 2024-09-19T06:11:53.953 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: btime 2024-09-19T06:09:26:797704+0000 2024-09-19T06:11:53.953 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T06:11:53.953 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T06:11:53.953 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: legacy client fscid: -1 2024-09-19T06:11:53.953 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: 2024-09-19T06:11:53.953 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: No filesystems configured 2024-09-19T06:11:53.953 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: mon.smithi155@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-19T06:11:53.953 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: mon.smithi155@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-19T06:11:53.953 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: mon.smithi155@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-19T06:11:53.953 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: mon.smithi155@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-19T06:11:53.953 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: mon.smithi155@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-19T06:11:53.953 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: mon.smithi155@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-19T06:11:53.953 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: mon.smithi155@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-09-19T06:11:53.954 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: mon.smithi155@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T06:11:53.954 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: mon.smithi155@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T06:11:53.954 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: mon.smithi155@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T06:11:53.954 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: mon.smithi155@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T06:11:53.954 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:53.954 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.954 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.954 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.954 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.954 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.954 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: Deploying daemon alertmanager.smithi053 on smithi053 2024-09-19T06:11:53.954 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='client.14189 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:11:53.954 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:53.954 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/1212914618' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T06:11:53.955 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:53.955 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/1212914618' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T06:11:53.955 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T06:11:53.955 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.955 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:53.955 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='client.14193 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi053:172.21.15.53=smithi053;smithi155:172.21.15.155=smithi155", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:11:53.955 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: Saving service mon spec with placement smithi053:172.21.15.53=smithi053;smithi155:172.21.15.155=smithi155;count:2 2024-09-19T06:11:53.955 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.955 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='client.? 172.21.15.155:0/997975947' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:53.955 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:53.955 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.955 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.955 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.955 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.955 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.956 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.956 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.956 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.956 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-19T06:11:53.956 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.956 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-19T06:11:53.956 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-19T06:11:53.956 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: Deploying daemon grafana.smithi053 on smithi053 2024-09-19T06:11:53.956 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.956 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:53.956 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='client.? 172.21.15.155:0/703382073' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:53.956 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:53.956 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='client.? 172.21.15.155:0/2254246009' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:53.956 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:53.957 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='client.? 172.21.15.155:0/1743455497' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:53.957 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:53.957 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='client.? 172.21.15.155:0/485025382' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:53.957 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:53.957 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.957 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.957 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.957 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.957 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.957 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.957 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.957 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.957 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: Deploying daemon prometheus.smithi053 on smithi053 2024-09-19T06:11:53.957 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:53.957 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.958 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='client.? 172.21.15.155:0/2763111410' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:53.958 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:53.958 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='client.? 172.21.15.155:0/3123150393' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:53.958 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:53.958 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='client.? 172.21.15.155:0/3736037119' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:53.958 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:53.958 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='client.? 172.21.15.155:0/4169899952' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:53.958 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:53.958 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:53.958 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='client.? 172.21.15.155:0/2894730809' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:53.958 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:53.958 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.958 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.958 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.958 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-19T06:11:53.959 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='client.? 172.21.15.155:0/3429379745' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:53.959 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14162 172.21.15.53:0/3045211634' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-19T06:11:53.959 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: mgrmap e14: smithi053.trakvg(active, since 53s) 2024-09-19T06:11:53.959 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='client.? 172.21.15.155:0/37779705' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:53.959 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='client.? 172.21.15.155:0/237523663' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:53.959 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: Active manager daemon smithi053.trakvg restarted 2024-09-19T06:11:53.959 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: Activating manager daemon smithi053.trakvg 2024-09-19T06:11:53.959 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: osdmap e5: 0 total, 0 up, 0 in 2024-09-19T06:11:53.959 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: mgrmap e15: smithi053.trakvg(active, starting, since 0.00322326s) 2024-09-19T06:11:53.959 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata", "id": "smithi053"}]: dispatch 2024-09-19T06:11:53.959 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mgr metadata", "who": "smithi053.trakvg", "id": "smithi053.trakvg"}]: dispatch 2024-09-19T06:11:53.959 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T06:11:53.959 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T06:11:53.959 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T06:11:53.959 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: Manager daemon smithi053.trakvg is now available 2024-09-19T06:11:53.960 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:11:53.960 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.960 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:11:53.960 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi053.trakvg/mirror_snapshot_schedule"}]: dispatch 2024-09-19T06:11:53.960 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi053.trakvg/trash_purge_schedule"}]: dispatch 2024-09-19T06:11:53.960 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.960 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: mgrmap e16: smithi053.trakvg(active, since 1.00536s) 2024-09-19T06:11:53.960 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.960 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: [19/Sep/2024:06:11:30] ENGINE Bus STARTING 2024-09-19T06:11:53.960 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='client.? 172.21.15.155:0/1576741719' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:53.960 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.960 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: [19/Sep/2024:06:11:30] ENGINE Serving on https://172.21.15.53:7150 2024-09-19T06:11:53.960 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: [19/Sep/2024:06:11:30] ENGINE Error in HTTPServer.serve 2024-09-19T06:11:53.960 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: Traceback (most recent call last): 2024-09-19T06:11:53.961 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T06:11:53.961 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: self._connections.run(self.expiration_interval) 2024-09-19T06:11:53.961 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T06:11:53.961 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: self._run(expiration_interval) 2024-09-19T06:11:53.961 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T06:11:53.961 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T06:11:53.961 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T06:11:53.961 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T06:11:53.961 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T06:11:53.961 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: s = self.context.wrap_socket( 2024-09-19T06:11:53.961 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T06:11:53.961 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: return self.sslsocket_class._create( 2024-09-19T06:11:53.961 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T06:11:53.961 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: self.do_handshake() 2024-09-19T06:11:53.962 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T06:11:53.962 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: self._sslobj.do_handshake() 2024-09-19T06:11:53.962 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T06:11:53.962 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: [19/Sep/2024:06:11:30] ENGINE Serving on http://172.21.15.53:8765 2024-09-19T06:11:53.962 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: [19/Sep/2024:06:11:30] ENGINE Bus STARTED 2024-09-19T06:11:53.962 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: mgrmap e17: smithi053.trakvg(active, since 2s) 2024-09-19T06:11:53.962 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.962 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.962 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.962 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.962 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd/host:smithi053", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:11:53.962 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='client.? 172.21.15.155:0/915497427' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:53.962 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.962 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.962 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.963 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.963 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd/host:smithi155", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:11:53.963 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:11:53.963 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:11:53.963 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: Updating smithi053:/etc/ceph/ceph.conf 2024-09-19T06:11:53.963 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: Updating smithi155:/etc/ceph/ceph.conf 2024-09-19T06:11:53.963 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: Updating smithi155:/var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:11:53.963 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: Updating smithi053:/var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:11:53.963 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.963 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.963 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.963 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.963 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.963 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi155", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-19T06:11:53.963 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi155", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-19T06:11:53.964 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:11:53.964 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:11:53.964 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:11:53.964 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: Updating smithi155:/var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:11:53.964 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: Updating smithi053:/var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:11:53.964 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: Deploying daemon ceph-exporter.smithi155 on smithi155 2024-09-19T06:11:53.964 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='client.? 172.21.15.155:0/172491323' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:53.964 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.964 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.964 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.964 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.964 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi155", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-19T06:11:53.964 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi155", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-09-19T06:11:53.964 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:11:53.965 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: Deploying daemon crash.smithi155 on smithi155 2024-09-19T06:11:53.965 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='client.? 172.21.15.155:0/2910809794' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:53.965 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.965 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.965 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.965 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.965 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: Deploying daemon node-exporter.smithi155 on smithi155 2024-09-19T06:11:53.965 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='client.? 172.21.15.155:0/1037565319' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:53.965 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.965 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:11:53.965 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='client.? 172.21.15.155:0/3780155486' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:53.965 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.965 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.965 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.966 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.966 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi155.piymgv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T06:11:53.966 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi155.piymgv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T06:11:53.966 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T06:11:53.966 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:11:53.966 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: Deploying daemon mgr.smithi155.piymgv on smithi155 2024-09-19T06:11:53.966 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:53.966 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.966 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='client.? 172.21.15.155:0/909868857' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:53.966 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.966 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.966 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.966 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.966 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T06:11:53.966 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:11:53.967 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: Deploying daemon mon.smithi155 on smithi155 2024-09-19T06:11:53.967 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:53.967 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:53.967 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='client.? 172.21.15.155:0/1993350318' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:53.967 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:53.967 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:53 smithi155 ceph-mon[35292]: mon.smithi155@-1(synchronizing).paxosservice(auth 1..8) refresh upgraded, format 0 -> 3 2024-09-19T06:11:54.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:53 smithi053 ceph-mon[27968]: from='client.? 172.21.15.155:0/1993350318' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:11:54.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:53 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:54.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:54 smithi155 bash[35261]: 043f44f8feb0f1dd9f903a987dd746c2ffce335378a7bf6f1f27939b6d1bc8eb 2024-09-19T06:11:54.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:54 smithi155 systemd[1]: Started Ceph mon.smithi155 for 9f2876d0-764d-11ef-bceb-c7b262605968. 2024-09-19T06:11:55.533 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-19T06:11:55.534 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T06:11:55.779 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi155/config 2024-09-19T06:11:58.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata", "id": "smithi053"}]: dispatch 2024-09-19T06:11:58.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: mon.smithi053 calling monitor election 2024-09-19T06:11:58.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata", "id": "smithi155"}]: dispatch 2024-09-19T06:11:58.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:58.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata", "id": "smithi155"}]: dispatch 2024-09-19T06:11:58.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: from='mgr.? 172.21.15.155:0/1309647630' entity='mgr.smithi155.piymgv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi155.piymgv/crt"}]: dispatch 2024-09-19T06:11:58.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata", "id": "smithi155"}]: dispatch 2024-09-19T06:11:58.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: mon.smithi155 calling monitor election 2024-09-19T06:11:58.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:58.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata", "id": "smithi155"}]: dispatch 2024-09-19T06:11:58.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata", "id": "smithi155"}]: dispatch 2024-09-19T06:11:58.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:58.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:11:58.986 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata", "id": "smithi155"}]: dispatch 2024-09-19T06:11:58.986 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: mon.smithi053 is new leader, mons smithi053,smithi155 in quorum (ranks 0,1) 2024-09-19T06:11:58.986 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: monmap epoch 2 2024-09-19T06:11:58.986 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: fsid 9f2876d0-764d-11ef-bceb-c7b262605968 2024-09-19T06:11:58.986 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: last_changed 2024-09-19T06:11:53.842920+0000 2024-09-19T06:11:58.986 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: created 2024-09-19T06:09:23.791881+0000 2024-09-19T06:11:58.986 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: min_mon_release 19 (squid) 2024-09-19T06:11:58.986 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: election_strategy: 1 2024-09-19T06:11:58.986 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: 0: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.smithi053 2024-09-19T06:11:58.986 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: 1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.smithi155 2024-09-19T06:11:58.986 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: fsmap 2024-09-19T06:11:58.986 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: osdmap e5: 0 total, 0 up, 0 in 2024-09-19T06:11:58.987 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: mgrmap e17: smithi053.trakvg(active, since 30s) 2024-09-19T06:11:58.987 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: overall HEALTH_OK 2024-09-19T06:11:58.987 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: Standby manager daemon smithi155.piymgv started 2024-09-19T06:11:58.987 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:58.987 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: from='mgr.? 172.21.15.155:0/1309647630' entity='mgr.smithi155.piymgv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:11:58.987 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: from='mgr.? 172.21.15.155:0/1309647630' entity='mgr.smithi155.piymgv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi155.piymgv/key"}]: dispatch 2024-09-19T06:11:58.987 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:58.987 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: from='mgr.? 172.21.15.155:0/1309647630' entity='mgr.smithi155.piymgv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:11:58.987 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:58.987 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:58.987 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:58 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:59.230 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.228+0000 7fac86234640 1 -- 172.21.15.155:0/3605992262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac68003530 msgr2=0x7fac680059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:59.230 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.228+0000 7fac86234640 1 --2- 172.21.15.155:0/3605992262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac68003530 0x7fac680059c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fac801016a0 tx=0x7fac7002bce0 comp rx=0 tx=0).stop 2024-09-19T06:11:59.230 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.228+0000 7fac86234640 1 -- 172.21.15.155:0/3605992262 shutdown_connections 2024-09-19T06:11:59.230 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.228+0000 7fac86234640 1 --2- 172.21.15.155:0/3605992262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac68003530 0x7fac680059c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:59.230 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.228+0000 7fac86234640 1 -- 172.21.15.155:0/3605992262 >> 172.21.15.155:0/3605992262 conn(0x7fac800fc220 msgr2=0x7fac800fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:59.230 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.228+0000 7fac86234640 1 -- 172.21.15.155:0/3605992262 shutdown_connections 2024-09-19T06:11:59.231 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.228+0000 7fac86234640 1 -- 172.21.15.155:0/3605992262 wait complete. 2024-09-19T06:11:59.231 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.229+0000 7fac86234640 1 Processor -- start 2024-09-19T06:11:59.232 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.229+0000 7fac86234640 1 -- start start 2024-09-19T06:11:59.232 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.229+0000 7fac86234640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac68003530 0x7fac80195f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:59.232 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.229+0000 7fac86234640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac80196440 0x7fac8019a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:59.232 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.229+0000 7fac86234640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fac80108520 con 0x7fac80196440 2024-09-19T06:11:59.232 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.229+0000 7fac86234640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fac801083a0 con 0x7fac68003530 2024-09-19T06:11:59.232 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.229+0000 7fac7f7fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac68003530 0x7fac80195f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:59.232 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.229+0000 7fac7f7fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac68003530 0x7fac80195f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:40516/0 (socket says 172.21.15.155:40516) 2024-09-19T06:11:59.233 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.229+0000 7fac7f7fe640 1 -- 172.21.15.155:0/2016241010 learned_addr learned my addr 172.21.15.155:0/2016241010 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-19T06:11:59.233 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.230+0000 7fac7f7fe640 1 -- 172.21.15.155:0/2016241010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac68003530 msgr2=0x7fac80195f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T06:11:59.233 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.230+0000 7fac7f7fe640 1 -- 172.21.15.155:0/2016241010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac68003530 msgr2=0x7fac80195f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T06:11:59.233 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.230+0000 7fac7f7fe640 1 --2- 172.21.15.155:0/2016241010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac68003530 0x7fac80195f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T06:11:59.233 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.230+0000 7fac7effd640 1 --2- 172.21.15.155:0/2016241010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac80196440 0x7fac8019a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:59.233 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.230+0000 7fac7f7fe640 1 --2- 172.21.15.155:0/2016241010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac68003530 0x7fac80195f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T06:11:59.233 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.230+0000 7fac7effd640 1 -- 172.21.15.155:0/2016241010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac68003530 msgr2=0x7fac80195f00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T06:11:59.233 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.230+0000 7fac7effd640 1 --2- 172.21.15.155:0/2016241010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac68003530 0x7fac80195f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:59.233 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.230+0000 7fac7effd640 1 -- 172.21.15.155:0/2016241010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac80196a60 con 0x7fac80196440 2024-09-19T06:11:59.233 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.230+0000 7fac7effd640 1 --2- 172.21.15.155:0/2016241010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac80196440 0x7fac8019a7d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fac7400b3d0 tx=0x7fac7400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:59.233 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.230+0000 7fac7cff9640 1 -- 172.21.15.155:0/2016241010 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fac74015020 con 0x7fac80196440 2024-09-19T06:11:59.233 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.230+0000 7fac7cff9640 1 -- 172.21.15.155:0/2016241010 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac74011ae0 con 0x7fac80196440 2024-09-19T06:11:59.233 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.230+0000 7fac86234640 1 -- 172.21.15.155:0/2016241010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac8019ad10 con 0x7fac80196440 2024-09-19T06:11:59.233 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.230+0000 7fac7cff9640 1 -- 172.21.15.155:0/2016241010 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fac74010c00 con 0x7fac80196440 2024-09-19T06:11:59.234 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.231+0000 7fac86234640 1 -- 172.21.15.155:0/2016241010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac8019b2d0 con 0x7fac80196440 2024-09-19T06:11:59.234 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.231+0000 7fac86234640 1 -- 172.21.15.155:0/2016241010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac8019b5e0 con 0x7fac80196440 2024-09-19T06:11:59.234 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.232+0000 7fac7cff9640 1 -- 172.21.15.155:0/2016241010 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 92495+0+0 (secure 0 0 0) 0x7fac74018020 con 0x7fac80196440 2024-09-19T06:11:59.234 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.233+0000 7fac7cff9640 1 --2- 172.21.15.155:0/2016241010 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fac6c06e330 0x7fac6c0707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:59.235 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.233+0000 7fac7cff9640 1 -- 172.21.15.155:0/2016241010 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fac7408cb80 con 0x7fac80196440 2024-09-19T06:11:59.235 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.233+0000 7fac7f7fe640 1 --2- 172.21.15.155:0/2016241010 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fac6c06e330 0x7fac6c0707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:59.235 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.233+0000 7fac7f7fe640 1 --2- 172.21.15.155:0/2016241010 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fac6c06e330 0x7fac6c0707f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fac70005dc0 tx=0x7fac70005d10 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:59.239 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.237+0000 7fac7cff9640 1 -- 172.21.15.155:0/2016241010 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fac7401f020 con 0x7fac80196440 2024-09-19T06:11:59.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata", "id": "smithi053"}]: dispatch 2024-09-19T06:11:59.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: mon.smithi053 calling monitor election 2024-09-19T06:11:59.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata", "id": "smithi155"}]: dispatch 2024-09-19T06:11:59.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:59.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata", "id": "smithi155"}]: dispatch 2024-09-19T06:11:59.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: from='mgr.? 172.21.15.155:0/1309647630' entity='mgr.smithi155.piymgv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi155.piymgv/crt"}]: dispatch 2024-09-19T06:11:59.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata", "id": "smithi155"}]: dispatch 2024-09-19T06:11:59.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: mon.smithi155 calling monitor election 2024-09-19T06:11:59.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:59.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata", "id": "smithi155"}]: dispatch 2024-09-19T06:11:59.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata", "id": "smithi155"}]: dispatch 2024-09-19T06:11:59.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:11:59.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:11:59.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata", "id": "smithi155"}]: dispatch 2024-09-19T06:11:59.357 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: mon.smithi053 is new leader, mons smithi053,smithi155 in quorum (ranks 0,1) 2024-09-19T06:11:59.357 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: monmap epoch 2 2024-09-19T06:11:59.357 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: fsid 9f2876d0-764d-11ef-bceb-c7b262605968 2024-09-19T06:11:59.357 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: last_changed 2024-09-19T06:11:53.842920+0000 2024-09-19T06:11:59.357 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: created 2024-09-19T06:09:23.791881+0000 2024-09-19T06:11:59.357 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: min_mon_release 19 (squid) 2024-09-19T06:11:59.357 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: election_strategy: 1 2024-09-19T06:11:59.357 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: 0: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.smithi053 2024-09-19T06:11:59.357 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: 1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.smithi155 2024-09-19T06:11:59.357 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: fsmap 2024-09-19T06:11:59.357 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: osdmap e5: 0 total, 0 up, 0 in 2024-09-19T06:11:59.357 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: mgrmap e17: smithi053.trakvg(active, since 30s) 2024-09-19T06:11:59.357 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: overall HEALTH_OK 2024-09-19T06:11:59.357 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: Standby manager daemon smithi155.piymgv started 2024-09-19T06:11:59.358 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:59.358 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: from='mgr.? 172.21.15.155:0/1309647630' entity='mgr.smithi155.piymgv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:11:59.358 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: from='mgr.? 172.21.15.155:0/1309647630' entity='mgr.smithi155.piymgv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi155.piymgv/key"}]: dispatch 2024-09-19T06:11:59.358 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:59.358 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: from='mgr.? 172.21.15.155:0/1309647630' entity='mgr.smithi155.piymgv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:11:59.358 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:59.358 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:59.358 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:58 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:11:59.482 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.480+0000 7fac86234640 1 -- 172.21.15.155:0/2016241010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fac80197260 con 0x7fac80196440 2024-09-19T06:11:59.482 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.481+0000 7fac7cff9640 1 -- 172.21.15.155:0/2016241010 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1031 (secure 0 0 0) 0x7fac7405b4d0 con 0x7fac80196440 2024-09-19T06:11:59.483 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:11:59.483 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":2,"fsid":"9f2876d0-764d-11ef-bceb-c7b262605968","modified":"2024-09-19T06:11:53.842920Z","created":"2024-09-19T06:09:23.791881Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi053","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi155","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:3300","nonce":0},{"type":"v1","addr":"172.21.15.155:6789","nonce":0}]},"addr":"172.21.15.155:6789/0","public_addr":"172.21.15.155:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T06:11:59.483 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 2 2024-09-19T06:11:59.485 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.483+0000 7fac86234640 1 -- 172.21.15.155:0/2016241010 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fac6c06e330 msgr2=0x7fac6c0707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:59.485 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.483+0000 7fac86234640 1 --2- 172.21.15.155:0/2016241010 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fac6c06e330 0x7fac6c0707f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fac70005dc0 tx=0x7fac70005d10 comp rx=0 tx=0).stop 2024-09-19T06:11:59.485 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.483+0000 7fac86234640 1 -- 172.21.15.155:0/2016241010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac80196440 msgr2=0x7fac8019a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:59.485 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.483+0000 7fac86234640 1 --2- 172.21.15.155:0/2016241010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac80196440 0x7fac8019a7d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fac7400b3d0 tx=0x7fac7400b890 comp rx=0 tx=0).stop 2024-09-19T06:11:59.485 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.483+0000 7fac86234640 1 -- 172.21.15.155:0/2016241010 shutdown_connections 2024-09-19T06:11:59.485 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.483+0000 7fac86234640 1 --2- 172.21.15.155:0/2016241010 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fac6c06e330 0x7fac6c0707f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:59.485 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.483+0000 7fac86234640 1 --2- 172.21.15.155:0/2016241010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac80196440 0x7fac8019a7d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:59.485 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.483+0000 7fac86234640 1 --2- 172.21.15.155:0/2016241010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac68003530 0x7fac80195f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:59.485 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.483+0000 7fac86234640 1 -- 172.21.15.155:0/2016241010 >> 172.21.15.155:0/2016241010 conn(0x7fac800fc220 msgr2=0x7fac800fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:59.485 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.483+0000 7fac86234640 1 -- 172.21.15.155:0/2016241010 shutdown_connections 2024-09-19T06:11:59.485 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:11:59.483+0000 7fac86234640 1 -- 172.21.15.155:0/2016241010 wait complete. 2024-09-19T06:12:00.212 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:59 smithi155 ceph-mon[35292]: mgrmap e18: smithi053.trakvg(active, since 30s), standbys: smithi155.piymgv 2024-09-19T06:12:00.212 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mgr metadata", "who": "smithi155.piymgv", "id": "smithi155.piymgv"}]: dispatch 2024-09-19T06:12:00.212 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:12:00.212 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:59 smithi155 ceph-mon[35292]: from='client.? 172.21.15.155:0/2016241010' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:12:00.213 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:11:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata", "id": "smithi155"}]: dispatch 2024-09-19T06:12:00.240 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-19T06:12:00.240 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-19T06:12:00.270 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:59 smithi053 ceph-mon[27968]: mgrmap e18: smithi053.trakvg(active, since 30s), standbys: smithi155.piymgv 2024-09-19T06:12:00.270 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mgr metadata", "who": "smithi155.piymgv", "id": "smithi155.piymgv"}]: dispatch 2024-09-19T06:12:00.271 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:12:00.271 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:59 smithi053 ceph-mon[27968]: from='client.? 172.21.15.155:0/2016241010' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:12:00.271 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:11:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata", "id": "smithi155"}]: dispatch 2024-09-19T06:12:00.515 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:12:00.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.936+0000 7f24eacab640 1 -- 172.21.15.53:0/2841824640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24e4102730 msgr2=0x7f24e4102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:00.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.936+0000 7f24eacab640 1 --2- 172.21.15.53:0/2841824640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24e4102730 0x7f24e4102b10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f24d4009810 tx=0x7f24d402a540 comp rx=0 tx=0).stop 2024-09-19T06:12:00.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.936+0000 7f24eacab640 1 -- 172.21.15.53:0/2841824640 shutdown_connections 2024-09-19T06:12:00.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.936+0000 7f24eacab640 1 --2- 172.21.15.53:0/2841824640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24e4102730 0x7f24e4102b10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.936+0000 7f24eacab640 1 -- 172.21.15.53:0/2841824640 >> 172.21.15.53:0/2841824640 conn(0x7f24e4077310 msgr2=0x7f24e40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:00.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.936+0000 7f24eacab640 1 -- 172.21.15.53:0/2841824640 shutdown_connections 2024-09-19T06:12:00.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.936+0000 7f24eacab640 1 -- 172.21.15.53:0/2841824640 wait complete. 2024-09-19T06:12:00.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.937+0000 7f24eacab640 1 Processor -- start 2024-09-19T06:12:00.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.937+0000 7f24eacab640 1 -- start start 2024-09-19T06:12:00.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.937+0000 7f24eacab640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24e4102730 0x7f24e4197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:00.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.937+0000 7f24eacab640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24e41975c0 0x7f24e419b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:00.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.937+0000 7f24eacab640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f24e410bb30 con 0x7f24e41975c0 2024-09-19T06:12:00.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.937+0000 7f24eacab640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f24e410b9b0 con 0x7f24e4102730 2024-09-19T06:12:00.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.937+0000 7f24e3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24e41975c0 0x7f24e419b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:00.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.937+0000 7f24e3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24e41975c0 0x7f24e419b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36480/0 (socket says 172.21.15.53:36480) 2024-09-19T06:12:00.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.937+0000 7f24e3fff640 1 -- 172.21.15.53:0/2470700987 learned_addr learned my addr 172.21.15.53:0/2470700987 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:12:00.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.937+0000 7f24e8a47640 1 --2- 172.21.15.53:0/2470700987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24e4102730 0x7f24e4197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:00.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.937+0000 7f24e8a47640 1 -- 172.21.15.53:0/2470700987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24e4102730 msgr2=0x7f24e4197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T06:12:00.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.937+0000 7f24e8a47640 1 -- 172.21.15.53:0/2470700987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24e4102730 msgr2=0x7f24e4197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T06:12:00.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.937+0000 7f24e8a47640 1 --2- 172.21.15.53:0/2470700987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24e4102730 0x7f24e4197080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T06:12:00.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.937+0000 7f24e8a47640 1 --2- 172.21.15.53:0/2470700987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24e4102730 0x7f24e4197080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T06:12:00.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.937+0000 7f24e3fff640 1 -- 172.21.15.53:0/2470700987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24e4102730 msgr2=0x7f24e4197080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T06:12:00.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.938+0000 7f24e3fff640 1 --2- 172.21.15.53:0/2470700987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24e4102730 0x7f24e4197080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.938+0000 7f24e3fff640 1 -- 172.21.15.53:0/2470700987 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e4197be0 con 0x7f24e41975c0 2024-09-19T06:12:00.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.938+0000 7f24e3fff640 1 --2- 172.21.15.53:0/2470700987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24e41975c0 0x7f24e419b950 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00b4a0 tx=0x7f24cc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:00.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.938+0000 7f24e1ffb640 1 -- 172.21.15.53:0/2470700987 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f24cc014070 con 0x7f24e41975c0 2024-09-19T06:12:00.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.938+0000 7f24e1ffb640 1 -- 172.21.15.53:0/2470700987 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24cc010ac0 con 0x7f24e41975c0 2024-09-19T06:12:00.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.938+0000 7f24eacab640 1 -- 172.21.15.53:0/2470700987 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e419be90 con 0x7f24e41975c0 2024-09-19T06:12:00.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.938+0000 7f24e1ffb640 1 -- 172.21.15.53:0/2470700987 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f24cc017c20 con 0x7f24e41975c0 2024-09-19T06:12:00.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.938+0000 7f24eacab640 1 -- 172.21.15.53:0/2470700987 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e419c3d0 con 0x7f24e41975c0 2024-09-19T06:12:00.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.940+0000 7f24eacab640 1 -- 172.21.15.53:0/2470700987 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24e4069f90 con 0x7f24e41975c0 2024-09-19T06:12:00.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.940+0000 7f24e1ffb640 1 -- 172.21.15.53:0/2470700987 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 92495+0+0 (secure 0 0 0) 0x7f24cc017070 con 0x7f24e41975c0 2024-09-19T06:12:00.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.940+0000 7f24e1ffb640 1 --2- 172.21.15.53:0/2470700987 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f24d006e380 0x7f24d0070840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:00.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.940+0000 7f24e1ffb640 1 -- 172.21.15.53:0/2470700987 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f24cc00d040 con 0x7f24e41975c0 2024-09-19T06:12:00.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.940+0000 7f24e8a47640 1 --2- 172.21.15.53:0/2470700987 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f24d006e380 0x7f24d0070840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:00.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.941+0000 7f24e8a47640 1 --2- 172.21.15.53:0/2470700987 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f24d006e380 0x7f24d0070840 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f24d40058e0 tx=0x7f24d4005870 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:00.946 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:00.945+0000 7f24e1ffb640 1 -- 172.21.15.53:0/2470700987 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f24cc01e030 con 0x7f24e41975c0 2024-09-19T06:12:01.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:01.118+0000 7f24eacab640 1 -- 172.21.15.53:0/2470700987 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f24e419c6e0 con 0x7f24e41975c0 2024-09-19T06:12:01.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:01.118+0000 7f24e1ffb640 1 -- 172.21.15.53:0/2470700987 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) ==== 76+0+225 (secure 0 0 0) 0x7f24cc05b1c0 con 0x7f24e41975c0 2024-09-19T06:12:01.120 INFO:teuthology.orchestra.run.smithi053.stdout:# minimal ceph.conf for 9f2876d0-764d-11ef-bceb-c7b262605968 2024-09-19T06:12:01.121 INFO:teuthology.orchestra.run.smithi053.stdout:[global] 2024-09-19T06:12:01.121 INFO:teuthology.orchestra.run.smithi053.stdout: fsid = 9f2876d0-764d-11ef-bceb-c7b262605968 2024-09-19T06:12:01.121 INFO:teuthology.orchestra.run.smithi053.stdout: mon_host = [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] 2024-09-19T06:12:01.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:01.120+0000 7f24eacab640 1 -- 172.21.15.53:0/2470700987 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f24d006e380 msgr2=0x7f24d0070840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:01.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:01.120+0000 7f24eacab640 1 --2- 172.21.15.53:0/2470700987 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f24d006e380 0x7f24d0070840 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f24d40058e0 tx=0x7f24d4005870 comp rx=0 tx=0).stop 2024-09-19T06:12:01.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:01.120+0000 7f24eacab640 1 -- 172.21.15.53:0/2470700987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24e41975c0 msgr2=0x7f24e419b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:01.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:01.120+0000 7f24eacab640 1 --2- 172.21.15.53:0/2470700987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24e41975c0 0x7f24e419b950 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00b4a0 tx=0x7f24cc00b960 comp rx=0 tx=0).stop 2024-09-19T06:12:01.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:01.120+0000 7f24eacab640 1 -- 172.21.15.53:0/2470700987 shutdown_connections 2024-09-19T06:12:01.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:01.120+0000 7f24eacab640 1 --2- 172.21.15.53:0/2470700987 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f24d006e380 0x7f24d0070840 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:01.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:01.120+0000 7f24eacab640 1 --2- 172.21.15.53:0/2470700987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24e41975c0 0x7f24e419b950 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:01.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:01.120+0000 7f24eacab640 1 --2- 172.21.15.53:0/2470700987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24e4102730 0x7f24e4197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:01.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:01.120+0000 7f24eacab640 1 -- 172.21.15.53:0/2470700987 >> 172.21.15.53:0/2470700987 conn(0x7f24e4077310 msgr2=0x7f24e4103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:01.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:01.120+0000 7f24eacab640 1 -- 172.21.15.53:0/2470700987 shutdown_connections 2024-09-19T06:12:01.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:01.120+0000 7f24eacab640 1 -- 172.21.15.53:0/2470700987 wait complete. 2024-09-19T06:12:01.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:00 smithi155 ceph-mon[35292]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:01.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:00 smithi053 ceph-mon[27968]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:01.829 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-19T06:12:01.830 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-19T06:12:01.830 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T06:12:01.897 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-19T06:12:01.897 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:12:01.963 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-19T06:12:01.963 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T06:12:01.990 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-19T06:12:01.990 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:12:02.056 INFO:tasks.cephadm:Deploying OSDs... 2024-09-19T06:12:02.056 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-19T06:12:02.056 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T06:12:02.073 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T06:12:02.073 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_1 2024-09-19T06:12:02.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:01 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/2470700987' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:02.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:01 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:02.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:01 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:02.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:01 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:02.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:01 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:12:02.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:01 smithi053 ceph-mon[27968]: Updating smithi053:/etc/ceph/ceph.conf 2024-09-19T06:12:02.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:01 smithi053 ceph-mon[27968]: Updating smithi155:/etc/ceph/ceph.conf 2024-09-19T06:12:02.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:01 smithi053 ceph-mon[27968]: Updating smithi155:/var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:12:02.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:01 smithi053 ceph-mon[27968]: Updating smithi053:/var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:12:02.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:01 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:02.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:01 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:02.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:01 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:02.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:01 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:02.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:01 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:02.121 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T06:12:02.121 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:12:02.121 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-19T06:12:02.122 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:12:02.122 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:12:02.122 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-09-19 06:11:22.090602477 +0000 2024-09-19T06:12:02.122 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-09-19 06:05:50.821356354 +0000 2024-09-19T06:12:02.122 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-09-19 06:05:50.821356354 +0000 2024-09-19T06:12:02.122 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-09-19 06:05:50.821356354 +0000 2024-09-19T06:12:02.122 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T06:12:02.188 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-09-19T06:12:02.188 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-09-19T06:12:02.188 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000117172 s, 4.4 MB/s 2024-09-19T06:12:02.190 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T06:12:02.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:01 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/2470700987' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:02.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:01 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:02.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:01 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:02.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:01 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:02.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:01 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:12:02.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:01 smithi155 ceph-mon[35292]: Updating smithi053:/etc/ceph/ceph.conf 2024-09-19T06:12:02.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:01 smithi155 ceph-mon[35292]: Updating smithi155:/etc/ceph/ceph.conf 2024-09-19T06:12:02.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:01 smithi155 ceph-mon[35292]: Updating smithi155:/var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:12:02.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:01 smithi155 ceph-mon[35292]: Updating smithi053:/var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:12:02.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:01 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:02.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:01 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:02.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:01 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:02.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:01 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:02.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:01 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:02.247 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_2 2024-09-19T06:12:02.303 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T06:12:02.303 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:12:02.303 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-19T06:12:02.303 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:12:02.303 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:12:02.303 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-09-19 06:11:22.090602477 +0000 2024-09-19T06:12:02.303 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-09-19 06:05:51.143350343 +0000 2024-09-19T06:12:02.303 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-09-19 06:05:51.143350343 +0000 2024-09-19T06:12:02.304 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-09-19 06:05:51.143350343 +0000 2024-09-19T06:12:02.304 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T06:12:02.368 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-09-19T06:12:02.368 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-09-19T06:12:02.368 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000107841 s, 4.7 MB/s 2024-09-19T06:12:02.369 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T06:12:02.426 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_3 2024-09-19T06:12:02.482 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T06:12:02.482 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:12:02.483 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-19T06:12:02.483 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:12:02.483 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:12:02.483 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-09-19 06:11:22.090602477 +0000 2024-09-19T06:12:02.483 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-09-19 06:05:51.481344032 +0000 2024-09-19T06:12:02.483 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-09-19 06:05:51.481344032 +0000 2024-09-19T06:12:02.483 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-09-19 06:05:51.481344032 +0000 2024-09-19T06:12:02.483 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T06:12:02.547 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-09-19T06:12:02.547 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-09-19T06:12:02.547 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 9.9925e-05 s, 5.1 MB/s 2024-09-19T06:12:02.548 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T06:12:02.607 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_4 2024-09-19T06:12:02.663 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T06:12:02.663 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:12:02.663 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-19T06:12:02.663 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:12:02.663 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:12:02.663 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-09-19 06:11:22.091602458 +0000 2024-09-19T06:12:02.663 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-09-19 06:05:51.806337965 +0000 2024-09-19T06:12:02.663 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-09-19 06:05:51.806337965 +0000 2024-09-19T06:12:02.663 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-09-19 06:05:51.806337965 +0000 2024-09-19T06:12:02.664 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T06:12:02.726 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-09-19T06:12:02.727 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-09-19T06:12:02.727 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 8.7487e-05 s, 5.9 MB/s 2024-09-19T06:12:02.728 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T06:12:02.784 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-19T06:12:02.785 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T06:12:02.801 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T06:12:02.802 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_1 2024-09-19T06:12:02.857 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T06:12:02.857 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:12:02.858 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-19T06:12:02.858 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:12:02.858 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:12:02.858 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-09-19 06:11:52.988391651 +0000 2024-09-19T06:12:02.858 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-09-19 06:05:48.174043968 +0000 2024-09-19T06:12:02.858 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-09-19 06:05:48.174043968 +0000 2024-09-19T06:12:02.858 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-09-19 06:05:48.174043968 +0000 2024-09-19T06:12:02.858 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T06:12:02.922 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-09-19T06:12:02.922 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-09-19T06:12:02.922 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000191476 s, 2.7 MB/s 2024-09-19T06:12:02.923 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T06:12:02.980 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_2 2024-09-19T06:12:03.036 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T06:12:03.036 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:12:03.036 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-19T06:12:03.036 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:12:03.036 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:12:03.037 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-09-19 06:11:52.988391651 +0000 2024-09-19T06:12:03.037 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-09-19 06:05:48.513036732 +0000 2024-09-19T06:12:03.037 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-09-19 06:05:48.513036732 +0000 2024-09-19T06:12:03.037 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-09-19 06:05:48.513036732 +0000 2024-09-19T06:12:03.037 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T06:12:03.103 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-09-19T06:12:03.103 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-09-19T06:12:03.103 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000177924 s, 2.9 MB/s 2024-09-19T06:12:03.105 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T06:12:03.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:02 smithi053 ceph-mon[27968]: Reconfiguring mon.smithi053 (unknown last config time)... 2024-09-19T06:12:03.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:02 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T06:12:03.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:02 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T06:12:03.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:02 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:03.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:02 smithi053 ceph-mon[27968]: Reconfiguring daemon mon.smithi053 on smithi053 2024-09-19T06:12:03.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:02 smithi053 ceph-mon[27968]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:03.162 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_3 2024-09-19T06:12:03.218 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T06:12:03.218 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:12:03.218 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-19T06:12:03.218 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:12:03.218 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:12:03.218 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-09-19 06:11:52.989391629 +0000 2024-09-19T06:12:03.218 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-09-19 06:05:48.845029647 +0000 2024-09-19T06:12:03.219 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-09-19 06:05:48.845029647 +0000 2024-09-19T06:12:03.219 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-09-19 06:05:48.845029647 +0000 2024-09-19T06:12:03.219 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T06:12:03.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:02 smithi155 ceph-mon[35292]: Reconfiguring mon.smithi053 (unknown last config time)... 2024-09-19T06:12:03.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:02 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T06:12:03.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:02 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T06:12:03.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:02 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:03.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:02 smithi155 ceph-mon[35292]: Reconfiguring daemon mon.smithi053 on smithi053 2024-09-19T06:12:03.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:02 smithi155 ceph-mon[35292]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:03.256 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-09-19T06:12:03.257 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-09-19T06:12:03.257 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.00019018 s, 2.7 MB/s 2024-09-19T06:12:03.258 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T06:12:03.314 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_4 2024-09-19T06:12:03.370 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T06:12:03.370 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:12:03.370 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-19T06:12:03.370 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:12:03.370 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:12:03.370 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-09-19 06:11:52.989391629 +0000 2024-09-19T06:12:03.370 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-09-19 06:05:49.164022838 +0000 2024-09-19T06:12:03.371 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-09-19 06:05:49.164022838 +0000 2024-09-19T06:12:03.371 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-09-19 06:05:49.164022838 +0000 2024-09-19T06:12:03.371 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T06:12:03.433 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-09-19T06:12:03.434 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-09-19T06:12:03.434 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000221203 s, 2.3 MB/s 2024-09-19T06:12:03.435 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T06:12:03.491 INFO:tasks.cephadm:Deploying osd.0 on smithi053 with /dev/vg_nvme/lv_4... 2024-09-19T06:12:03.492 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T06:12:04.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:04 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:04.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:04 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:04.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:04 smithi053 ceph-mon[27968]: Reconfiguring mgr.smithi053.trakvg (unknown last config time)... 2024-09-19T06:12:04.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:04 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi053.trakvg", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T06:12:04.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:04 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T06:12:04.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:04 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:04.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:04 smithi053 ceph-mon[27968]: Reconfiguring daemon mgr.smithi053.trakvg on smithi053 2024-09-19T06:12:04.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:04 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:04.697 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:12:04.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:04 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:04.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:04 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:04.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:04 smithi155 ceph-mon[35292]: Reconfiguring mgr.smithi053.trakvg (unknown last config time)... 2024-09-19T06:12:04.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:04 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi053.trakvg", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T06:12:04.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:04 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T06:12:04.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:04 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:04.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:04 smithi155 ceph-mon[35292]: Reconfiguring daemon mgr.smithi053.trakvg on smithi053 2024-09-19T06:12:04.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:04 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:05.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:05 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:05.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:05 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:05.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:05 smithi155 ceph-mon[35292]: Reconfiguring ceph-exporter.smithi053 (monmap changed)... 2024-09-19T06:12:05.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:05 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi053", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-19T06:12:05.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:05 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:05.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:05 smithi155 ceph-mon[35292]: Reconfiguring daemon ceph-exporter.smithi053 on smithi053 2024-09-19T06:12:05.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:05 smithi155 ceph-mon[35292]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:06.014 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:05 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:06.014 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:05 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:06.014 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:05 smithi053 ceph-mon[27968]: Reconfiguring ceph-exporter.smithi053 (monmap changed)... 2024-09-19T06:12:06.014 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:05 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi053", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-19T06:12:06.014 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:05 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:06.015 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:05 smithi053 ceph-mon[27968]: Reconfiguring daemon ceph-exporter.smithi053 on smithi053 2024-09-19T06:12:06.015 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:05 smithi053 ceph-mon[27968]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:07.825 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:07 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:07.825 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:07 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:07.825 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:07 smithi053 ceph-mon[27968]: Reconfiguring crash.smithi053 (monmap changed)... 2024-09-19T06:12:07.825 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:07 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi053", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-19T06:12:07.825 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:07 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:07.825 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:07 smithi053 ceph-mon[27968]: Reconfiguring daemon crash.smithi053 on smithi053 2024-09-19T06:12:07.825 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:07 smithi053 ceph-mon[27968]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:07.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:07 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:07.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:07 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:07.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:07 smithi155 ceph-mon[35292]: Reconfiguring crash.smithi053 (monmap changed)... 2024-09-19T06:12:07.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:07 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi053", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-19T06:12:07.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:07 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:07.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:07 smithi155 ceph-mon[35292]: Reconfiguring daemon crash.smithi053 on smithi053 2024-09-19T06:12:07.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:07 smithi155 ceph-mon[35292]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:08.104 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:12:08.139 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_4 2024-09-19T06:12:08.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:08 smithi155 ceph-mon[35292]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:09.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:08 smithi053 ceph-mon[27968]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:09.393 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:12:09.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.802+0000 7fee2b5fb640 1 -- 172.21.15.53:0/4124354355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee24101960 msgr2=0x7fee24101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:09.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.802+0000 7fee2b5fb640 1 --2- 172.21.15.53:0/4124354355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee24101960 0x7fee24101d60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fee1c009880 tx=0x7fee1c02a4a0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.802+0000 7fee2b5fb640 1 -- 172.21.15.53:0/4124354355 shutdown_connections 2024-09-19T06:12:09.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.802+0000 7fee2b5fb640 1 --2- 172.21.15.53:0/4124354355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee24102b60 0x7fee24102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.802+0000 7fee2b5fb640 1 --2- 172.21.15.53:0/4124354355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee24101960 0x7fee24101d60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.802+0000 7fee2b5fb640 1 -- 172.21.15.53:0/4124354355 >> 172.21.15.53:0/4124354355 conn(0x7fee240fd0f0 msgr2=0x7fee240ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:09.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.802+0000 7fee2b5fb640 1 -- 172.21.15.53:0/4124354355 shutdown_connections 2024-09-19T06:12:09.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.802+0000 7fee2b5fb640 1 -- 172.21.15.53:0/4124354355 wait complete. 2024-09-19T06:12:09.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.803+0000 7fee2b5fb640 1 Processor -- start 2024-09-19T06:12:09.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.803+0000 7fee2b5fb640 1 -- start start 2024-09-19T06:12:09.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.803+0000 7fee2b5fb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee24101960 0x7fee241974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:09.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.803+0000 7fee2b5fb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee24102b60 0x7fee24197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:09.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.803+0000 7fee2b5fb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fee2410a770 con 0x7fee24102b60 2024-09-19T06:12:09.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.803+0000 7fee2b5fb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fee2410a5f0 con 0x7fee24101960 2024-09-19T06:12:09.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.803+0000 7fee28b96640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee24102b60 0x7fee24197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:09.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.803+0000 7fee28b96640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee24102b60 0x7fee24197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35398/0 (socket says 172.21.15.53:35398) 2024-09-19T06:12:09.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.803+0000 7fee28b96640 1 -- 172.21.15.53:0/405873526 learned_addr learned my addr 172.21.15.53:0/405873526 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:12:09.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.803+0000 7fee29397640 1 --2- 172.21.15.53:0/405873526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee24101960 0x7fee241974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:09.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.803+0000 7fee28b96640 1 -- 172.21.15.53:0/405873526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee24101960 msgr2=0x7fee241974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:09.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.803+0000 7fee28b96640 1 --2- 172.21.15.53:0/405873526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee24101960 0x7fee241974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.803+0000 7fee28b96640 1 -- 172.21.15.53:0/405873526 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee24198010 con 0x7fee24102b60 2024-09-19T06:12:09.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.804+0000 7fee28b96640 1 --2- 172.21.15.53:0/405873526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee24102b60 0x7fee24197a20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fee1800b420 tx=0x7fee1800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:09.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.804+0000 7fee227fc640 1 -- 172.21.15.53:0/405873526 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fee18011630 con 0x7fee24102b60 2024-09-19T06:12:09.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.804+0000 7fee227fc640 1 -- 172.21.15.53:0/405873526 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee18016070 con 0x7fee24102b60 2024-09-19T06:12:09.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.804+0000 7fee2b5fb640 1 -- 172.21.15.53:0/405873526 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee2419ca80 con 0x7fee24102b60 2024-09-19T06:12:09.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.804+0000 7fee227fc640 1 -- 172.21.15.53:0/405873526 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fee18010c90 con 0x7fee24102b60 2024-09-19T06:12:09.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.804+0000 7fee2b5fb640 1 -- 172.21.15.53:0/405873526 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee2419d060 con 0x7fee24102b60 2024-09-19T06:12:09.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.805+0000 7fee2b5fb640 1 -- 172.21.15.53:0/405873526 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee24101de0 con 0x7fee24102b60 2024-09-19T06:12:09.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.805+0000 7fee227fc640 1 -- 172.21.15.53:0/405873526 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 92495+0+0 (secure 0 0 0) 0x7fee180117d0 con 0x7fee24102b60 2024-09-19T06:12:09.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.806+0000 7fee227fc640 1 --2- 172.21.15.53:0/405873526 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fee0c06e230 0x7fee0c0706f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:09.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.806+0000 7fee227fc640 1 -- 172.21.15.53:0/405873526 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fee1808c8d0 con 0x7fee24102b60 2024-09-19T06:12:09.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.806+0000 7fee29397640 1 --2- 172.21.15.53:0/405873526 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fee0c06e230 0x7fee0c0706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:09.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.806+0000 7fee29397640 1 --2- 172.21.15.53:0/405873526 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fee0c06e230 0x7fee0c0706f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fee1c000c00 tx=0x7fee1c004e20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:09.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.810+0000 7fee227fc640 1 -- 172.21.15.53:0/405873526 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fee1805b280 con 0x7fee24102b60 2024-09-19T06:12:10.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:09.999+0000 7fee2b5fb640 1 -- 172.21.15.53:0/405873526 --> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fee2402c640 con 0x7fee0c06e230 2024-09-19T06:12:10.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:10 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:10.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:10 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:10.484 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:10 smithi155 ceph-mon[35292]: Reconfiguring alertmanager.smithi053 (dependencies changed)... 2024-09-19T06:12:10.484 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:10 smithi155 ceph-mon[35292]: Reconfiguring daemon alertmanager.smithi053 on smithi053 2024-09-19T06:12:10.484 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:10 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:12:10.484 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:10 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:12:10.484 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:10 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:10.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:10 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:10.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:10 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:10.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:10 smithi053 ceph-mon[27968]: Reconfiguring alertmanager.smithi053 (dependencies changed)... 2024-09-19T06:12:10.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:10 smithi053 ceph-mon[27968]: Reconfiguring daemon alertmanager.smithi053 on smithi053 2024-09-19T06:12:10.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:10 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:12:10.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:10 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:12:10.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:10 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:11.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:11 smithi053 ceph-mon[27968]: from='client.14258 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:12:11.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:11 smithi053 ceph-mon[27968]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:11.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:11 smithi155 ceph-mon[35292]: from='client.14258 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:12:11.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:11 smithi155 ceph-mon[35292]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:12.939 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:12 smithi053 ceph-mon[27968]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:13.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:12 smithi155 ceph-mon[35292]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:13.972 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:13 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:12:13.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:13 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:12:14.987 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:14 smithi053 ceph-mon[27968]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:15.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:14 smithi155 ceph-mon[35292]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:16.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:16 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:16.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:16 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:16.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:16 smithi155 ceph-mon[35292]: Reconfiguring grafana.smithi053 (dependencies changed)... 2024-09-19T06:12:16.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:16 smithi155 ceph-mon[35292]: Reconfiguring daemon grafana.smithi053 on smithi053 2024-09-19T06:12:16.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:16 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:16.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:16 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:16.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:16 smithi053 ceph-mon[27968]: Reconfiguring grafana.smithi053 (dependencies changed)... 2024-09-19T06:12:16.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:16 smithi053 ceph-mon[27968]: Reconfiguring daemon grafana.smithi053 on smithi053 2024-09-19T06:12:17.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:17 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/1939489808' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35349d4a-0bde-46f6-8d5f-a019f0f89095"}]: dispatch 2024-09-19T06:12:17.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:17 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/1939489808' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35349d4a-0bde-46f6-8d5f-a019f0f89095"}]': finished 2024-09-19T06:12:17.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:17 smithi053 ceph-mon[27968]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T06:12:17.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:17 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:12:17.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:17 smithi053 ceph-mon[27968]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:17.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:17 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/4248735784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:12:17.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:17 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/1939489808' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35349d4a-0bde-46f6-8d5f-a019f0f89095"}]: dispatch 2024-09-19T06:12:17.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:17 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/1939489808' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35349d4a-0bde-46f6-8d5f-a019f0f89095"}]': finished 2024-09-19T06:12:17.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:17 smithi155 ceph-mon[35292]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T06:12:17.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:17 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:12:17.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:17 smithi155 ceph-mon[35292]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:17.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:17 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/4248735784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:12:18.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:18 smithi155 ceph-mon[35292]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:19.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:18 smithi053 ceph-mon[27968]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:21.089 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:20 smithi053 ceph-mon[27968]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:21.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:20 smithi155 ceph-mon[35292]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:23.100 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:22 smithi053 ceph-mon[27968]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:23.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:22 smithi155 ceph-mon[35292]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:24.985 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:24 smithi053 ceph-mon[27968]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:25.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:24 smithi155 ceph-mon[35292]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:26.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:26 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:26.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:26 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:26.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:26 smithi155 ceph-mon[35292]: Reconfiguring prometheus.smithi053 (dependencies changed)... 2024-09-19T06:12:27.039 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:26 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:27.039 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:26 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:27.040 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:26 smithi053 ceph-mon[27968]: Reconfiguring prometheus.smithi053 (dependencies changed)... 2024-09-19T06:12:27.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:27 smithi155 ceph-mon[35292]: Reconfiguring daemon prometheus.smithi053 on smithi053 2024-09-19T06:12:27.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:27 smithi155 ceph-mon[35292]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:27.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:27 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T06:12:27.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:27 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:28.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:27 smithi053 ceph-mon[27968]: Reconfiguring daemon prometheus.smithi053 on smithi053 2024-09-19T06:12:28.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:27 smithi053 ceph-mon[27968]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:28.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:27 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T06:12:28.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:27 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:28.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:28 smithi155 ceph-mon[35292]: Deploying daemon osd.0 on smithi053 2024-09-19T06:12:29.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:28 smithi053 ceph-mon[27968]: Deploying daemon osd.0 on smithi053 2024-09-19T06:12:29.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:29 smithi053 ceph-mon[27968]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:29.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:12:30.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:29 smithi155 ceph-mon[35292]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:30.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:29 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:12:31.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:31 smithi053 ceph-mon[27968]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:31.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:31 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:31.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:31 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:31.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:31 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi155", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-19T06:12:31.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:31 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:31.994 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:31 smithi155 ceph-mon[35292]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:31.994 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:31 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:31.994 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:31 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:31.994 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:31 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi155", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-19T06:12:31.994 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:31 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:32.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:32 smithi053 ceph-mon[27968]: Reconfiguring ceph-exporter.smithi155 (monmap changed)... 2024-09-19T06:12:32.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:32 smithi053 ceph-mon[27968]: Reconfiguring daemon ceph-exporter.smithi155 on smithi155 2024-09-19T06:12:32.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:32 smithi155 ceph-mon[35292]: Reconfiguring ceph-exporter.smithi155 (monmap changed)... 2024-09-19T06:12:32.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:32 smithi155 ceph-mon[35292]: Reconfiguring daemon ceph-exporter.smithi155 on smithi155 2024-09-19T06:12:33.852 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:33 smithi053 ceph-mon[27968]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:33.852 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:33 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:33.852 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:33 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:33.852 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:33 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi155", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-19T06:12:33.852 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:33 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:33.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:33 smithi155 ceph-mon[35292]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:33.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:33 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:33.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:33 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:33.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:33 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi155", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-19T06:12:33.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:33 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:34.840 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:34 smithi053 ceph-mon[27968]: Reconfiguring crash.smithi155 (monmap changed)... 2024-09-19T06:12:34.840 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:34 smithi053 ceph-mon[27968]: Reconfiguring daemon crash.smithi155 on smithi155 2024-09-19T06:12:34.840 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:34 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:34.841 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:34 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:34.841 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:34 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi155.piymgv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T06:12:34.841 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:34 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T06:12:34.841 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:34 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:34.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:34 smithi155 ceph-mon[35292]: Reconfiguring crash.smithi155 (monmap changed)... 2024-09-19T06:12:34.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:34 smithi155 ceph-mon[35292]: Reconfiguring daemon crash.smithi155 on smithi155 2024-09-19T06:12:34.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:34 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:34.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:34 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:34.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:34 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi155.piymgv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T06:12:34.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:34 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T06:12:34.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:34 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:36.044 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:35 smithi155 ceph-mon[35292]: Reconfiguring mgr.smithi155.piymgv (monmap changed)... 2024-09-19T06:12:36.044 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:35 smithi155 ceph-mon[35292]: Reconfiguring daemon mgr.smithi155.piymgv on smithi155 2024-09-19T06:12:36.044 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:35 smithi155 ceph-mon[35292]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:36.044 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:35 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:36.044 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:35 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:36.044 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:35 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T06:12:36.044 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:35 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T06:12:36.044 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:35 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:36.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:35 smithi053 ceph-mon[27968]: Reconfiguring mgr.smithi155.piymgv (monmap changed)... 2024-09-19T06:12:36.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:35 smithi053 ceph-mon[27968]: Reconfiguring daemon mgr.smithi155.piymgv on smithi155 2024-09-19T06:12:36.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:35 smithi053 ceph-mon[27968]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:36.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:35 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:36.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:35 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:36.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:35 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T06:12:36.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:35 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T06:12:36.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:35 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:36.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:36 smithi053 ceph-mon[27968]: Reconfiguring mon.smithi155 (monmap changed)... 2024-09-19T06:12:36.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:36 smithi053 ceph-mon[27968]: Reconfiguring daemon mon.smithi155 on smithi155 2024-09-19T06:12:37.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:36 smithi155 ceph-mon[35292]: Reconfiguring mon.smithi155 (monmap changed)... 2024-09-19T06:12:37.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:36 smithi155 ceph-mon[35292]: Reconfiguring daemon mon.smithi155 on smithi155 2024-09-19T06:12:37.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:37 smithi053 ceph-mon[27968]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:37.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:37 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:37.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:37 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:37.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:37 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-19T06:12:37.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:37 smithi053 ceph-mon[27968]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-19T06:12:37.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:37 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi053.front.sepia.ceph.com:9093"}]: dispatch 2024-09-19T06:12:37.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:37 smithi053 ceph-mon[27968]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi053.front.sepia.ceph.com:9093"}]: dispatch 2024-09-19T06:12:37.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:37 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:37.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:37 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-19T06:12:37.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:37 smithi053 ceph-mon[27968]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-19T06:12:37.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:37 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi053.front.sepia.ceph.com:3000"}]: dispatch 2024-09-19T06:12:37.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:37 smithi053 ceph-mon[27968]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi053.front.sepia.ceph.com:3000"}]: dispatch 2024-09-19T06:12:37.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:37 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:37.857 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:37 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-19T06:12:37.857 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:37 smithi053 ceph-mon[27968]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-19T06:12:37.857 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:37 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi053.front.sepia.ceph.com:9095"}]: dispatch 2024-09-19T06:12:37.857 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:37 smithi053 ceph-mon[27968]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi053.front.sepia.ceph.com:9095"}]: dispatch 2024-09-19T06:12:37.857 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:37 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:37.857 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:37 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:12:37.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:37 smithi155 ceph-mon[35292]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:37.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:37 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:37.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:37 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:37.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:37 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-19T06:12:37.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:37 smithi155 ceph-mon[35292]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-19T06:12:37.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:37 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi053.front.sepia.ceph.com:9093"}]: dispatch 2024-09-19T06:12:37.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:37 smithi155 ceph-mon[35292]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi053.front.sepia.ceph.com:9093"}]: dispatch 2024-09-19T06:12:37.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:37 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:37.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:37 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-19T06:12:37.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:37 smithi155 ceph-mon[35292]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-19T06:12:37.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:37 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi053.front.sepia.ceph.com:3000"}]: dispatch 2024-09-19T06:12:37.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:37 smithi155 ceph-mon[35292]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi053.front.sepia.ceph.com:3000"}]: dispatch 2024-09-19T06:12:37.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:37 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:37.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:37 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-19T06:12:37.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:37 smithi155 ceph-mon[35292]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-19T06:12:37.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:37 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi053.front.sepia.ceph.com:9095"}]: dispatch 2024-09-19T06:12:37.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:37 smithi155 ceph-mon[35292]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi053.front.sepia.ceph.com:9095"}]: dispatch 2024-09-19T06:12:37.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:37 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:37.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:37 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:12:38.970 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:38 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:38.970 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:38 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:38.970 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:38 smithi155 ceph-mon[35292]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:38.997 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:38 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:38.997 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:38 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:38.998 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:38 smithi053 ceph-mon[27968]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:40.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:39 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:40.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:39 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:40.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:39 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:40.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:39 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:41.087 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:40 smithi053 ceph-mon[27968]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:41.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:40 smithi155 ceph-mon[35292]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:41.568 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 0 on host 'smithi053' 2024-09-19T06:12:41.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:41.566+0000 7fee227fc640 1 -- 172.21.15.53:0/405873526 <== mgr.14217 v2:172.21.15.53:6800/3884355752 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fee2402c640 con 0x7fee0c06e230 2024-09-19T06:12:41.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:41.568+0000 7fee2b5fb640 1 -- 172.21.15.53:0/405873526 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fee0c06e230 msgr2=0x7fee0c0706f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:41.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:41.568+0000 7fee2b5fb640 1 --2- 172.21.15.53:0/405873526 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fee0c06e230 0x7fee0c0706f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fee1c000c00 tx=0x7fee1c004e20 comp rx=0 tx=0).stop 2024-09-19T06:12:41.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:41.568+0000 7fee2b5fb640 1 -- 172.21.15.53:0/405873526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee24102b60 msgr2=0x7fee24197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:41.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:41.568+0000 7fee2b5fb640 1 --2- 172.21.15.53:0/405873526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee24102b60 0x7fee24197a20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fee1800b420 tx=0x7fee1800b8e0 comp rx=0 tx=0).stop 2024-09-19T06:12:41.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:41.568+0000 7fee2b5fb640 1 -- 172.21.15.53:0/405873526 shutdown_connections 2024-09-19T06:12:41.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:41.568+0000 7fee2b5fb640 1 --2- 172.21.15.53:0/405873526 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fee0c06e230 0x7fee0c0706f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:41.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:41.568+0000 7fee2b5fb640 1 --2- 172.21.15.53:0/405873526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee24102b60 0x7fee24197a20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:41.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:41.568+0000 7fee2b5fb640 1 --2- 172.21.15.53:0/405873526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee24101960 0x7fee241974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:41.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:41.568+0000 7fee2b5fb640 1 -- 172.21.15.53:0/405873526 >> 172.21.15.53:0/405873526 conn(0x7fee240fd0f0 msgr2=0x7fee24106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:41.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:41.568+0000 7fee2b5fb640 1 -- 172.21.15.53:0/405873526 shutdown_connections 2024-09-19T06:12:41.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:41.568+0000 7fee2b5fb640 1 -- 172.21.15.53:0/405873526 wait complete. 2024-09-19T06:12:42.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:42 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:42.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:42 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:42.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:42 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:42.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:42 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:43.065 DEBUG:teuthology.orchestra.run.smithi053:osd.0> sudo journalctl -f -n 0 -u ceph-9f2876d0-764d-11ef-bceb-c7b262605968@osd.0.service 2024-09-19T06:12:43.068 INFO:tasks.cephadm:Deploying osd.1 on smithi053 with /dev/vg_nvme/lv_3... 2024-09-19T06:12:43.068 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T06:12:43.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:43 smithi053 ceph-mon[27968]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:43.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:43 smithi053 ceph-mon[27968]: from='osd.0 [v2:172.21.15.53:6802/1456825097,v1:172.21.15.53:6803/1456825097]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T06:12:43.878 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:12:43.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:43 smithi155 ceph-mon[35292]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:43.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:43 smithi155 ceph-mon[35292]: from='osd.0 [v2:172.21.15.53:6802/1456825097,v1:172.21.15.53:6803/1456825097]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T06:12:44.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:44 smithi053 ceph-mon[27968]: from='osd.0 [v2:172.21.15.53:6802/1456825097,v1:172.21.15.53:6803/1456825097]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T06:12:44.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:44 smithi053 ceph-mon[27968]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T06:12:44.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:44 smithi053 ceph-mon[27968]: from='osd.0 [v2:172.21.15.53:6802/1456825097,v1:172.21.15.53:6803/1456825097]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-19T06:12:44.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:44 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:12:44.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:44 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:12:44.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:44 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:44.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:44 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:44.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:44 smithi155 ceph-mon[35292]: from='osd.0 [v2:172.21.15.53:6802/1456825097,v1:172.21.15.53:6803/1456825097]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T06:12:44.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:44 smithi155 ceph-mon[35292]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T06:12:44.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:44 smithi155 ceph-mon[35292]: from='osd.0 [v2:172.21.15.53:6802/1456825097,v1:172.21.15.53:6803/1456825097]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-19T06:12:44.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:44 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:12:44.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:44 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:12:44.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:44 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:44.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:44 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:45.105 INFO:journalctl@ceph.osd.0.smithi053.stdout:Sep 19 06:12:44 smithi053 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-0[46154]: 2024-09-19T06:12:44.841+0000 7f707756c640 -1 osd.0 0 waiting for initial osdmap 2024-09-19T06:12:45.105 INFO:journalctl@ceph.osd.0.smithi053.stdout:Sep 19 06:12:44 smithi053 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-0[46154]: 2024-09-19T06:12:44.843+0000 7f706f38c640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:12:45.679 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:45 smithi053 ceph-mon[27968]: from='osd.0 [v2:172.21.15.53:6802/1456825097,v1:172.21.15.53:6803/1456825097]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-19T06:12:45.680 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:45 smithi053 ceph-mon[27968]: osdmap e8: 1 total, 0 up, 1 in 2024-09-19T06:12:45.680 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:45 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:12:45.680 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:45 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:12:45.680 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:45 smithi053 ceph-mon[27968]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:45.680 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:45 smithi053 ceph-mon[27968]: from='osd.0 [v2:172.21.15.53:6802/1456825097,v1:172.21.15.53:6803/1456825097]' entity='osd.0' 2024-09-19T06:12:45.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:45 smithi155 ceph-mon[35292]: from='osd.0 [v2:172.21.15.53:6802/1456825097,v1:172.21.15.53:6803/1456825097]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-19T06:12:45.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:45 smithi155 ceph-mon[35292]: osdmap e8: 1 total, 0 up, 1 in 2024-09-19T06:12:45.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:45 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:12:45.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:45 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:12:45.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:45 smithi155 ceph-mon[35292]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:12:45.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:45 smithi155 ceph-mon[35292]: from='osd.0 [v2:172.21.15.53:6802/1456825097,v1:172.21.15.53:6803/1456825097]' entity='osd.0' 2024-09-19T06:12:46.591 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:12:46.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:46 smithi053 ceph-mon[27968]: purged_snaps scrub starts 2024-09-19T06:12:46.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:46 smithi053 ceph-mon[27968]: purged_snaps scrub ok 2024-09-19T06:12:46.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:46 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:12:46.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:46 smithi053 ceph-mon[27968]: osd.0 [v2:172.21.15.53:6802/1456825097,v1:172.21.15.53:6803/1456825097] boot 2024-09-19T06:12:46.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:46 smithi053 ceph-mon[27968]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T06:12:46.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:46 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:12:46.620 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_3 2024-09-19T06:12:46.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:46 smithi155 ceph-mon[35292]: purged_snaps scrub starts 2024-09-19T06:12:46.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:46 smithi155 ceph-mon[35292]: purged_snaps scrub ok 2024-09-19T06:12:46.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:46 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:12:46.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:46 smithi155 ceph-mon[35292]: osd.0 [v2:172.21.15.53:6802/1456825097,v1:172.21.15.53:6803/1456825097] boot 2024-09-19T06:12:46.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:46 smithi155 ceph-mon[35292]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T06:12:46.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:46 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:12:47.027 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:12:47.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.637+0000 7f8f7ef43640 1 -- 172.21.15.53:0/809993025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f78102b60 msgr2=0x7f8f78102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:47.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.637+0000 7f8f7ef43640 1 --2- 172.21.15.53:0/809993025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f78102b60 0x7f8f78102fe0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8f64009880 tx=0x7f8f6402a4a0 comp rx=0 tx=0).stop 2024-09-19T06:12:47.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.637+0000 7f8f7ef43640 1 -- 172.21.15.53:0/809993025 shutdown_connections 2024-09-19T06:12:47.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.637+0000 7f8f7ef43640 1 --2- 172.21.15.53:0/809993025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f78102b60 0x7f8f78102fe0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:47.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.637+0000 7f8f7ef43640 1 --2- 172.21.15.53:0/809993025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f78101960 0x7f8f78101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:47.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.637+0000 7f8f7ef43640 1 -- 172.21.15.53:0/809993025 >> 172.21.15.53:0/809993025 conn(0x7f8f780fd0f0 msgr2=0x7f8f780ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:47.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.637+0000 7f8f7ef43640 1 -- 172.21.15.53:0/809993025 shutdown_connections 2024-09-19T06:12:47.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.637+0000 7f8f7ef43640 1 -- 172.21.15.53:0/809993025 wait complete. 2024-09-19T06:12:47.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.638+0000 7f8f7ef43640 1 Processor -- start 2024-09-19T06:12:47.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.638+0000 7f8f7ef43640 1 -- start start 2024-09-19T06:12:47.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.638+0000 7f8f7ef43640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f78101960 0x7f8f78197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:47.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.638+0000 7f8f7ef43640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f78102b60 0x7f8f78197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:47.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.638+0000 7f8f7ef43640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7810a790 con 0x7f8f78101960 2024-09-19T06:12:47.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.638+0000 7f8f7ef43640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7810a610 con 0x7f8f78102b60 2024-09-19T06:12:47.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.638+0000 7f8f7ccdf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f78101960 0x7f8f78197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:47.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.638+0000 7f8f7ccdf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f78101960 0x7f8f78197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50570/0 (socket says 172.21.15.53:50570) 2024-09-19T06:12:47.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.638+0000 7f8f7ccdf640 1 -- 172.21.15.53:0/3851801407 learned_addr learned my addr 172.21.15.53:0/3851801407 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:12:47.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.638+0000 7f8f77fff640 1 --2- 172.21.15.53:0/3851801407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f78102b60 0x7f8f78197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:47.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.639+0000 7f8f7ccdf640 1 -- 172.21.15.53:0/3851801407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f78102b60 msgr2=0x7f8f78197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:47.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.639+0000 7f8f7ccdf640 1 --2- 172.21.15.53:0/3851801407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f78102b60 0x7f8f78197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:47.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.639+0000 7f8f7ccdf640 1 -- 172.21.15.53:0/3851801407 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f78198050 con 0x7f8f78101960 2024-09-19T06:12:47.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.639+0000 7f8f7ccdf640 1 --2- 172.21.15.53:0/3851801407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f78101960 0x7f8f78197520 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c00b3d0 tx=0x7f8f6c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:47.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.639+0000 7f8f75ffb640 1 -- 172.21.15.53:0/3851801407 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8f6c015020 con 0x7f8f78101960 2024-09-19T06:12:47.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.639+0000 7f8f75ffb640 1 -- 172.21.15.53:0/3851801407 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f6c011ae0 con 0x7f8f78101960 2024-09-19T06:12:47.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.639+0000 7f8f75ffb640 1 -- 172.21.15.53:0/3851801407 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8f6c010c80 con 0x7f8f78101960 2024-09-19T06:12:47.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.639+0000 7f8f7ef43640 1 -- 172.21.15.53:0/3851801407 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f7819cac0 con 0x7f8f78101960 2024-09-19T06:12:47.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.639+0000 7f8f7ef43640 1 -- 172.21.15.53:0/3851801407 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f7819d020 con 0x7f8f78101960 2024-09-19T06:12:47.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.640+0000 7f8f7ef43640 1 -- 172.21.15.53:0/3851801407 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f78101de0 con 0x7f8f78101960 2024-09-19T06:12:47.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.641+0000 7f8f75ffb640 1 -- 172.21.15.53:0/3851801407 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 92495+0+0 (secure 0 0 0) 0x7f8f6c018020 con 0x7f8f78101960 2024-09-19T06:12:47.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.642+0000 7f8f75ffb640 1 --2- 172.21.15.53:0/3851801407 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f8f6806e280 0x7f8f68070740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:47.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.642+0000 7f8f75ffb640 1 -- 172.21.15.53:0/3851801407 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1982+0+0 (secure 0 0 0) 0x7f8f6c08cd70 con 0x7f8f78101960 2024-09-19T06:12:47.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.642+0000 7f8f77fff640 1 --2- 172.21.15.53:0/3851801407 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f8f6806e280 0x7f8f68070740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:47.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.643+0000 7f8f77fff640 1 --2- 172.21.15.53:0/3851801407 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f8f6806e280 0x7f8f68070740 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8f781989d0 tx=0x7f8f640095f0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:47.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.647+0000 7f8f75ffb640 1 -- 172.21.15.53:0/3851801407 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8f6c01f020 con 0x7f8f78101960 2024-09-19T06:12:47.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:12:47.832+0000 7f8f7ef43640 1 -- 172.21.15.53:0/3851801407 --> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f8f78062630 con 0x7f8f6806e280 2024-09-19T06:12:47.834 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:47 smithi053 ceph-mon[27968]: pgmap v36: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:12:47.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:47 smithi155 ceph-mon[35292]: pgmap v36: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:12:48.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:48 smithi053 ceph-mon[27968]: osdmap e10: 1 total, 1 up, 1 in 2024-09-19T06:12:48.906 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:48 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:12:48.906 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:48 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:12:48.906 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:48 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:48.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:48 smithi155 ceph-mon[35292]: osdmap e10: 1 total, 1 up, 1 in 2024-09-19T06:12:48.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:48 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:12:48.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:48 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:12:48.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:48 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:49.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:49 smithi155 ceph-mon[35292]: from='client.14276 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:12:49.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:49 smithi155 ceph-mon[35292]: pgmap v38: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:12:49.799 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:49 smithi053 ceph-mon[27968]: from='client.14276 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:12:49.800 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:49 smithi053 ceph-mon[27968]: pgmap v38: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:12:51.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:50 smithi053 ceph-mon[27968]: Detected new or changed devices on smithi053 2024-09-19T06:12:51.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:50 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:51.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:50 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:51.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:50 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:12:51.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:50 smithi053 ceph-mon[27968]: Adjusting osd_memory_target on smithi053 to 11844M 2024-09-19T06:12:51.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:50 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:51.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:50 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:51.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:50 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:12:51.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:50 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:51.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:50 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:12:51.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:50 smithi053 ceph-mon[27968]: pgmap v39: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:12:51.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:50 smithi155 ceph-mon[35292]: Detected new or changed devices on smithi053 2024-09-19T06:12:51.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:50 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:51.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:50 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:51.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:50 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:12:51.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:50 smithi155 ceph-mon[35292]: Adjusting osd_memory_target on smithi053 to 11844M 2024-09-19T06:12:51.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:50 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:51.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:50 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:51.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:50 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:12:51.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:50 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:51.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:50 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:12:51.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:50 smithi155 ceph-mon[35292]: pgmap v39: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:12:52.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:51 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/4028697478' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "832c2772-a460-4691-8db7-ad1d0c48cc9f"}]: dispatch 2024-09-19T06:12:52.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:51 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/4028697478' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "832c2772-a460-4691-8db7-ad1d0c48cc9f"}]': finished 2024-09-19T06:12:52.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:51 smithi053 ceph-mon[27968]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T06:12:52.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:51 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:12:52.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:51 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/616909001' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:12:52.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:51 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:52.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:51 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:52.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:51 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/4028697478' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "832c2772-a460-4691-8db7-ad1d0c48cc9f"}]: dispatch 2024-09-19T06:12:52.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:51 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/4028697478' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "832c2772-a460-4691-8db7-ad1d0c48cc9f"}]': finished 2024-09-19T06:12:52.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:51 smithi155 ceph-mon[35292]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T06:12:52.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:51 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:12:52.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:51 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/616909001' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:12:52.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:51 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:52.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:51 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:53.029 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:52 smithi053 ceph-mon[27968]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:12:53.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:52 smithi155 ceph-mon[35292]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:12:55.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:55 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:55.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:55 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:55.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:55 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:55.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:55 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:12:55.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:55 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:55.484 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:55 smithi155 ceph-mon[35292]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:12:55.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:55 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:55.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:55 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:55.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:55 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:55.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:55 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:12:55.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:55 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:12:55.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:55 smithi053 ceph-mon[27968]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:12:57.109 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:56 smithi053 ceph-mon[27968]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:12:57.244 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:56 smithi155 ceph-mon[35292]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:12:58.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:58 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T06:12:58.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:58 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:58.844 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:58 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T06:12:58.844 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:58 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:59.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:59 smithi155 ceph-mon[35292]: Deploying daemon osd.1 on smithi053 2024-09-19T06:12:59.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:59 smithi155 ceph-mon[35292]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:12:59.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:12:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:12:59.852 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:59 smithi053 ceph-mon[27968]: Deploying daemon osd.1 on smithi053 2024-09-19T06:12:59.852 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:59 smithi053 ceph-mon[27968]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:12:59.852 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:12:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:13:01.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:00 smithi053 ceph-mon[27968]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:13:01.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:00 smithi155 ceph-mon[35292]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:13:03.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:02 smithi053 ceph-mon[27968]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:13:03.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:02 smithi155 ceph-mon[35292]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:13:04.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:04 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:13:04.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:04 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:04.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:04 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:04.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:04 smithi053 ceph-mon[27968]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:13:05.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:04 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:13:05.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:04 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:05.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:04 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:05.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:04 smithi155 ceph-mon[35292]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:13:07.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:07 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:07.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:07 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:07.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:07 smithi053 ceph-mon[27968]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:13:07.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:07.924+0000 7f8f75ffb640 1 -- 172.21.15.53:0/3851801407 <== mgr.14217 v2:172.21.15.53:6800/3884355752 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8f78062630 con 0x7f8f6806e280 2024-09-19T06:13:07.927 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 1 on host 'smithi053' 2024-09-19T06:13:07.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:07.927+0000 7f8f7ef43640 1 -- 172.21.15.53:0/3851801407 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f8f6806e280 msgr2=0x7f8f68070740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:07.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:07.927+0000 7f8f7ef43640 1 --2- 172.21.15.53:0/3851801407 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f8f6806e280 0x7f8f68070740 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8f781989d0 tx=0x7f8f640095f0 comp rx=0 tx=0).stop 2024-09-19T06:13:07.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:07.927+0000 7f8f7ef43640 1 -- 172.21.15.53:0/3851801407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f78101960 msgr2=0x7f8f78197520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:07.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:07.927+0000 7f8f7ef43640 1 --2- 172.21.15.53:0/3851801407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f78101960 0x7f8f78197520 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c00b3d0 tx=0x7f8f6c00b890 comp rx=0 tx=0).stop 2024-09-19T06:13:07.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:07.927+0000 7f8f7ef43640 1 -- 172.21.15.53:0/3851801407 shutdown_connections 2024-09-19T06:13:07.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:07.927+0000 7f8f7ef43640 1 --2- 172.21.15.53:0/3851801407 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f8f6806e280 0x7f8f68070740 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:07.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:07.927+0000 7f8f7ef43640 1 --2- 172.21.15.53:0/3851801407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f78102b60 0x7f8f78197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:07.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:07.927+0000 7f8f7ef43640 1 --2- 172.21.15.53:0/3851801407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f78101960 0x7f8f78197520 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:07.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:07.927+0000 7f8f7ef43640 1 -- 172.21.15.53:0/3851801407 >> 172.21.15.53:0/3851801407 conn(0x7f8f780fd0f0 msgr2=0x7f8f78106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:07.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:07.927+0000 7f8f7ef43640 1 -- 172.21.15.53:0/3851801407 shutdown_connections 2024-09-19T06:13:07.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:07.927+0000 7f8f7ef43640 1 -- 172.21.15.53:0/3851801407 wait complete. 2024-09-19T06:13:07.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:07 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:07.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:07 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:07.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:07 smithi155 ceph-mon[35292]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:13:08.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:08 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:08.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:08 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:08.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:08 smithi155 ceph-mon[35292]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:13:09.173 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:08 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:09.173 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:08 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:09.173 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:08 smithi053 ceph-mon[27968]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:13:09.199 DEBUG:teuthology.orchestra.run.smithi053:osd.1> sudo journalctl -f -n 0 -u ceph-9f2876d0-764d-11ef-bceb-c7b262605968@osd.1.service 2024-09-19T06:13:09.203 INFO:tasks.cephadm:Deploying osd.2 on smithi053 with /dev/vg_nvme/lv_2... 2024-09-19T06:13:09.203 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T06:13:09.710 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 19 06:13:09 smithi053 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-1[52663]: 2024-09-19T06:13:09.483+0000 7f945527e640 -1 osd.1 0 log_to_monitors true 2024-09-19T06:13:09.907 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:13:10.029 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:09 smithi053 ceph-mon[27968]: from='osd.1 [v2:172.21.15.53:6810/3638097004,v1:172.21.15.53:6811/3638097004]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T06:13:10.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:09 smithi155 ceph-mon[35292]: from='osd.1 [v2:172.21.15.53:6810/3638097004,v1:172.21.15.53:6811/3638097004]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T06:13:11.201 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:10 smithi053 ceph-mon[27968]: from='osd.1 [v2:172.21.15.53:6810/3638097004,v1:172.21.15.53:6811/3638097004]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T06:13:11.201 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:10 smithi053 ceph-mon[27968]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T06:13:11.201 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:10 smithi053 ceph-mon[27968]: from='osd.1 [v2:172.21.15.53:6810/3638097004,v1:172.21.15.53:6811/3638097004]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-19T06:13:11.201 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:10 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:13:11.201 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:10 smithi053 ceph-mon[27968]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:13:11.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:10 smithi155 ceph-mon[35292]: from='osd.1 [v2:172.21.15.53:6810/3638097004,v1:172.21.15.53:6811/3638097004]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T06:13:11.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:10 smithi155 ceph-mon[35292]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T06:13:11.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:10 smithi155 ceph-mon[35292]: from='osd.1 [v2:172.21.15.53:6810/3638097004,v1:172.21.15.53:6811/3638097004]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-19T06:13:11.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:10 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:13:11.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:10 smithi155 ceph-mon[35292]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:13:11.551 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 19 06:13:11 smithi053 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-1[52663]: 2024-09-19T06:13:11.196+0000 7f9451a15640 -1 osd.1 0 waiting for initial osdmap 2024-09-19T06:13:11.551 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 19 06:13:11 smithi053 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-1[52663]: 2024-09-19T06:13:11.199+0000 7f9449021640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:13:12.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:11 smithi053 ceph-mon[27968]: from='osd.1 [v2:172.21.15.53:6810/3638097004,v1:172.21.15.53:6811/3638097004]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-19T06:13:12.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:11 smithi053 ceph-mon[27968]: osdmap e13: 2 total, 1 up, 2 in 2024-09-19T06:13:12.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:11 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:13:12.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:11 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:13:12.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:11 smithi053 ceph-mon[27968]: from='osd.1 [v2:172.21.15.53:6810/3638097004,v1:172.21.15.53:6811/3638097004]' entity='osd.1' 2024-09-19T06:13:12.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:11 smithi155 ceph-mon[35292]: from='osd.1 [v2:172.21.15.53:6810/3638097004,v1:172.21.15.53:6811/3638097004]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-19T06:13:12.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:11 smithi155 ceph-mon[35292]: osdmap e13: 2 total, 1 up, 2 in 2024-09-19T06:13:12.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:11 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:13:12.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:11 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:13:12.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:11 smithi155 ceph-mon[35292]: from='osd.1 [v2:172.21.15.53:6810/3638097004,v1:172.21.15.53:6811/3638097004]' entity='osd.1' 2024-09-19T06:13:13.229 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:12 smithi053 ceph-mon[27968]: purged_snaps scrub starts 2024-09-19T06:13:13.229 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:12 smithi053 ceph-mon[27968]: purged_snaps scrub ok 2024-09-19T06:13:13.229 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:12 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:13:13.230 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:12 smithi053 ceph-mon[27968]: osd.1 [v2:172.21.15.53:6810/3638097004,v1:172.21.15.53:6811/3638097004] boot 2024-09-19T06:13:13.230 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:12 smithi053 ceph-mon[27968]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T06:13:13.230 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:12 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:13:13.230 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:12 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:13.230 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:12 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:13.230 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:12 smithi053 ceph-mon[27968]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:13:13.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:12 smithi155 ceph-mon[35292]: purged_snaps scrub starts 2024-09-19T06:13:13.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:12 smithi155 ceph-mon[35292]: purged_snaps scrub ok 2024-09-19T06:13:13.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:12 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:13:13.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:12 smithi155 ceph-mon[35292]: osd.1 [v2:172.21.15.53:6810/3638097004,v1:172.21.15.53:6811/3638097004] boot 2024-09-19T06:13:13.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:12 smithi155 ceph-mon[35292]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T06:13:13.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:12 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:13:13.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:12 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:13.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:12 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:13.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:12 smithi155 ceph-mon[35292]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:13:13.323 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:13:13.348 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_2 2024-09-19T06:13:14.459 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:14 smithi053 ceph-mon[27968]: osdmap e15: 2 total, 2 up, 2 in 2024-09-19T06:13:14.459 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:14 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:13:14.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:14 smithi155 ceph-mon[35292]: osdmap e15: 2 total, 2 up, 2 in 2024-09-19T06:13:14.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:14 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:13:14.989 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:13:15.262 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:15 smithi053 ceph-mon[27968]: pgmap v56: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:13:15.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.475+0000 7ff15ccfa640 1 -- 172.21.15.53:0/214332157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff158104d20 msgr2=0x7ff158107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:15.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.475+0000 7ff15ccfa640 1 --2- 172.21.15.53:0/214332157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff158104d20 0x7ff158107110 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff1400098b0 tx=0x7ff14002a4d0 comp rx=0 tx=0).stop 2024-09-19T06:13:15.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.475+0000 7ff15ccfa640 1 -- 172.21.15.53:0/214332157 shutdown_connections 2024-09-19T06:13:15.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.475+0000 7ff15ccfa640 1 --2- 172.21.15.53:0/214332157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff158104d20 0x7ff158107110 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:15.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.475+0000 7ff15ccfa640 1 --2- 172.21.15.53:0/214332157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1581023f0 0x7ff1581047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:15.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.475+0000 7ff15ccfa640 1 -- 172.21.15.53:0/214332157 >> 172.21.15.53:0/214332157 conn(0x7ff1580fc010 msgr2=0x7ff1580fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:15.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.476+0000 7ff15ccfa640 1 -- 172.21.15.53:0/214332157 shutdown_connections 2024-09-19T06:13:15.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.476+0000 7ff15ccfa640 1 -- 172.21.15.53:0/214332157 wait complete. 2024-09-19T06:13:15.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.476+0000 7ff15ccfa640 1 Processor -- start 2024-09-19T06:13:15.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.476+0000 7ff15ccfa640 1 -- start start 2024-09-19T06:13:15.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.476+0000 7ff15ccfa640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1581023f0 0x7ff158197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:15.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.476+0000 7ff15ccfa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff158104d20 0x7ff1581979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:15.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.477+0000 7ff15ccfa640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff15810a700 con 0x7ff158104d20 2024-09-19T06:13:15.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.477+0000 7ff15ccfa640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff15810a580 con 0x7ff1581023f0 2024-09-19T06:13:15.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.477+0000 7ff14ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff158104d20 0x7ff1581979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:15.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.477+0000 7ff14ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff158104d20 0x7ff1581979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55294/0 (socket says 172.21.15.53:55294) 2024-09-19T06:13:15.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.477+0000 7ff14ffff640 1 -- 172.21.15.53:0/1094454617 learned_addr learned my addr 172.21.15.53:0/1094454617 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:13:15.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.477+0000 7ff15659c640 1 --2- 172.21.15.53:0/1094454617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1581023f0 0x7ff158197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:15.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.477+0000 7ff14ffff640 1 -- 172.21.15.53:0/1094454617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1581023f0 msgr2=0x7ff158197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:15.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.477+0000 7ff14ffff640 1 --2- 172.21.15.53:0/1094454617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1581023f0 0x7ff158197470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:15.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.477+0000 7ff14ffff640 1 -- 172.21.15.53:0/1094454617 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff158197fa0 con 0x7ff158104d20 2024-09-19T06:13:15.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.477+0000 7ff14ffff640 1 --2- 172.21.15.53:0/1094454617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff158104d20 0x7ff1581979b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff140009230 tx=0x7ff1400045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:15.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.477+0000 7ff14f7fe640 1 -- 172.21.15.53:0/1094454617 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff14003f070 con 0x7ff158104d20 2024-09-19T06:13:15.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.477+0000 7ff14f7fe640 1 -- 172.21.15.53:0/1094454617 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1400049d0 con 0x7ff158104d20 2024-09-19T06:13:15.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.477+0000 7ff15ccfa640 1 -- 172.21.15.53:0/1094454617 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff15819c9b0 con 0x7ff158104d20 2024-09-19T06:13:15.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.477+0000 7ff14f7fe640 1 -- 172.21.15.53:0/1094454617 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff140035030 con 0x7ff158104d20 2024-09-19T06:13:15.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.477+0000 7ff15ccfa640 1 -- 172.21.15.53:0/1094454617 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff15819cf30 con 0x7ff158104d20 2024-09-19T06:13:15.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.479+0000 7ff15ccfa640 1 -- 172.21.15.53:0/1094454617 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1581027d0 con 0x7ff158104d20 2024-09-19T06:13:15.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.479+0000 7ff14f7fe640 1 -- 172.21.15.53:0/1094454617 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 92495+0+0 (secure 0 0 0) 0x7ff140042070 con 0x7ff158104d20 2024-09-19T06:13:15.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.479+0000 7ff14f7fe640 1 --2- 172.21.15.53:0/1094454617 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7ff14406e280 0x7ff144070740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:15.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.480+0000 7ff14f7fe640 1 -- 172.21.15.53:0/1094454617 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2414+0+0 (secure 0 0 0) 0x7ff1400adfe0 con 0x7ff158104d20 2024-09-19T06:13:15.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.480+0000 7ff15659c640 1 --2- 172.21.15.53:0/1094454617 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7ff14406e280 0x7ff144070740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:15.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.480+0000 7ff15659c640 1 --2- 172.21.15.53:0/1094454617 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7ff14406e280 0x7ff144070740 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff148005fd0 tx=0x7ff148005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:15.485 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:15 smithi155 ceph-mon[35292]: pgmap v56: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:13:15.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.484+0000 7ff14f7fe640 1 -- 172.21.15.53:0/1094454617 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff14007c600 con 0x7ff158104d20 2024-09-19T06:13:15.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:15.674+0000 7ff15ccfa640 1 -- 172.21.15.53:0/1094454617 --> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff1581055c0 con 0x7ff14406e280 2024-09-19T06:13:16.453 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:16 smithi053 ceph-mon[27968]: from='client.14294 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:13:16.453 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:16 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:13:16.453 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:16 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:13:16.453 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:16 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:13:16.453 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:16 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:16.453 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:16 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:16.453 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:16 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:16.453 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:16 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:16.453 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:16 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:16.454 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:16 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:13:16.454 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:16 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:13:16.454 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:16 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:16.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:16 smithi155 ceph-mon[35292]: from='client.14294 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:13:16.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:16 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:13:16.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:16 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:13:16.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:16 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:13:16.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:16 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:16.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:16 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:16.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:16 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:16.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:16 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:16.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:16 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:16.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:16 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:13:16.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:16 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:13:16.484 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:16 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:17.409 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:17 smithi053 ceph-mon[27968]: Detected new or changed devices on smithi053 2024-09-19T06:13:17.409 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:17 smithi053 ceph-mon[27968]: Adjusting osd_memory_target on smithi053 to 5922M 2024-09-19T06:13:17.409 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:17 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:13:17.409 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:17 smithi053 ceph-mon[27968]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:13:17.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:17 smithi155 ceph-mon[35292]: Detected new or changed devices on smithi053 2024-09-19T06:13:17.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:17 smithi155 ceph-mon[35292]: Adjusting osd_memory_target on smithi053 to 5922M 2024-09-19T06:13:17.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:17 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:13:17.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:17 smithi155 ceph-mon[35292]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:13:19.358 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:19 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:19.358 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:19 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:19.358 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:19 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/2249236105' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57282117-559f-4495-b85f-90b68b2439ad"}]: dispatch 2024-09-19T06:13:19.358 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:19 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/2249236105' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57282117-559f-4495-b85f-90b68b2439ad"}]': finished 2024-09-19T06:13:19.359 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:19 smithi053 ceph-mon[27968]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T06:13:19.359 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:19 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:13:19.359 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:19 smithi053 ceph-mon[27968]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:13:19.359 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:19 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/1872318131' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:13:19.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:19 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:19.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:19 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:19.484 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:19 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/2249236105' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57282117-559f-4495-b85f-90b68b2439ad"}]: dispatch 2024-09-19T06:13:19.484 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:19 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/2249236105' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57282117-559f-4495-b85f-90b68b2439ad"}]': finished 2024-09-19T06:13:19.484 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:19 smithi155 ceph-mon[35292]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T06:13:19.484 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:19 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:13:19.484 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:19 smithi155 ceph-mon[35292]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:13:19.484 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:19 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/1872318131' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:13:21.030 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:20 smithi053 ceph-mon[27968]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:13:21.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:20 smithi155 ceph-mon[35292]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:13:22.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:22 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:22.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:22 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:22.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:22 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:13:22.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:22 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:13:22.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:22 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:22.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:22 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:22.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:22 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:22.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:22 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:13:22.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:22 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:13:22.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:22 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:23.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:23 smithi053 ceph-mon[27968]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:13:23.700 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:23 smithi155 ceph-mon[35292]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:13:25.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:24 smithi053 ceph-mon[27968]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:13:25.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:24 smithi155 ceph-mon[35292]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:13:26.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:25 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T06:13:26.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:25 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:13:26.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:25 smithi053 ceph-mon[27968]: Deploying daemon osd.2 on smithi053 2024-09-19T06:13:26.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:25 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T06:13:26.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:25 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:13:26.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:25 smithi155 ceph-mon[35292]: Deploying daemon osd.2 on smithi053 2024-09-19T06:13:27.046 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:26 smithi053 ceph-mon[27968]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:13:27.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:26 smithi155 ceph-mon[35292]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:13:28.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:28 smithi155 ceph-mon[35292]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:13:28.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:28 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:13:29.036 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:28 smithi053 ceph-mon[27968]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:13:29.036 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:28 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:13:30.944 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:30 smithi053 ceph-mon[27968]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:13:31.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:30 smithi155 ceph-mon[35292]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:13:32.073 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:31 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:13:32.073 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:31 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:32.073 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:31 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:32.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:31 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:13:32.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:31 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:32.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:31 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:33.019 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:32 smithi053 ceph-mon[27968]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:13:33.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:32 smithi155 ceph-mon[35292]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:13:35.043 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:34 smithi053 ceph-mon[27968]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:13:35.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:34 smithi155 ceph-mon[35292]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:13:35.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:35.324+0000 7ff14f7fe640 1 -- 172.21.15.53:0/1094454617 <== mgr.14217 v2:172.21.15.53:6800/3884355752 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff1581055c0 con 0x7ff14406e280 2024-09-19T06:13:35.326 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 2 on host 'smithi053' 2024-09-19T06:13:35.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:35.327+0000 7ff15ccfa640 1 -- 172.21.15.53:0/1094454617 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7ff14406e280 msgr2=0x7ff144070740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:35.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:35.327+0000 7ff15ccfa640 1 --2- 172.21.15.53:0/1094454617 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7ff14406e280 0x7ff144070740 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff148005fd0 tx=0x7ff148005850 comp rx=0 tx=0).stop 2024-09-19T06:13:35.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:35.327+0000 7ff15ccfa640 1 -- 172.21.15.53:0/1094454617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff158104d20 msgr2=0x7ff1581979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:35.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:35.327+0000 7ff15ccfa640 1 --2- 172.21.15.53:0/1094454617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff158104d20 0x7ff1581979b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff140009230 tx=0x7ff1400045e0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:35.327+0000 7ff15ccfa640 1 -- 172.21.15.53:0/1094454617 shutdown_connections 2024-09-19T06:13:35.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:35.327+0000 7ff15ccfa640 1 --2- 172.21.15.53:0/1094454617 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7ff14406e280 0x7ff144070740 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:35.327+0000 7ff15ccfa640 1 --2- 172.21.15.53:0/1094454617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff158104d20 0x7ff1581979b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:35.327+0000 7ff15ccfa640 1 --2- 172.21.15.53:0/1094454617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1581023f0 0x7ff158197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:35.327+0000 7ff15ccfa640 1 -- 172.21.15.53:0/1094454617 >> 172.21.15.53:0/1094454617 conn(0x7ff1580fc010 msgr2=0x7ff1580fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:35.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:35.327+0000 7ff15ccfa640 1 -- 172.21.15.53:0/1094454617 shutdown_connections 2024-09-19T06:13:35.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:35.327+0000 7ff15ccfa640 1 -- 172.21.15.53:0/1094454617 wait complete. 2024-09-19T06:13:36.505 DEBUG:teuthology.orchestra.run.smithi053:osd.2> sudo journalctl -f -n 0 -u ceph-9f2876d0-764d-11ef-bceb-c7b262605968@osd.2.service 2024-09-19T06:13:36.507 INFO:tasks.cephadm:Deploying osd.3 on smithi053 with /dev/vg_nvme/lv_1... 2024-09-19T06:13:36.508 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T06:13:36.529 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:36 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:36.529 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:36 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:36.529 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:36 smithi053 ceph-mon[27968]: from='osd.2 [v2:172.21.15.53:6818/1988602192,v1:172.21.15.53:6819/1988602192]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T06:13:36.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:36 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:36.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:36 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:36.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:36 smithi155 ceph-mon[35292]: from='osd.2 [v2:172.21.15.53:6818/1988602192,v1:172.21.15.53:6819/1988602192]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T06:13:37.207 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:13:37.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:37 smithi053 ceph-mon[27968]: from='osd.2 [v2:172.21.15.53:6818/1988602192,v1:172.21.15.53:6819/1988602192]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T06:13:37.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:37 smithi053 ceph-mon[27968]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T06:13:37.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:37 smithi053 ceph-mon[27968]: from='osd.2 [v2:172.21.15.53:6818/1988602192,v1:172.21.15.53:6819/1988602192]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-19T06:13:37.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:37 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:13:37.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:37 smithi053 ceph-mon[27968]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:13:37.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:37 smithi155 ceph-mon[35292]: from='osd.2 [v2:172.21.15.53:6818/1988602192,v1:172.21.15.53:6819/1988602192]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T06:13:37.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:37 smithi155 ceph-mon[35292]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T06:13:37.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:37 smithi155 ceph-mon[35292]: from='osd.2 [v2:172.21.15.53:6818/1988602192,v1:172.21.15.53:6819/1988602192]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-19T06:13:37.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:37 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:13:37.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:37 smithi155 ceph-mon[35292]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:13:38.105 INFO:journalctl@ceph.osd.2.smithi053.stdout:Sep 19 06:13:37 smithi053 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-2[59344]: 2024-09-19T06:13:37.615+0000 7fde65254640 -1 osd.2 0 waiting for initial osdmap 2024-09-19T06:13:38.105 INFO:journalctl@ceph.osd.2.smithi053.stdout:Sep 19 06:13:37 smithi053 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-2[59344]: 2024-09-19T06:13:37.618+0000 7fde5d074640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:13:38.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:38 smithi053 ceph-mon[27968]: from='osd.2 [v2:172.21.15.53:6818/1988602192,v1:172.21.15.53:6819/1988602192]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-19T06:13:38.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:38 smithi053 ceph-mon[27968]: osdmap e18: 3 total, 2 up, 3 in 2024-09-19T06:13:38.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:38 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:13:38.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:38 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:13:38.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:38 smithi053 ceph-mon[27968]: from='osd.2 [v2:172.21.15.53:6818/1988602192,v1:172.21.15.53:6819/1988602192]' entity='osd.2' 2024-09-19T06:13:38.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:38 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:13:38.634 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:38 smithi155 ceph-mon[35292]: from='osd.2 [v2:172.21.15.53:6818/1988602192,v1:172.21.15.53:6819/1988602192]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-19T06:13:38.634 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:38 smithi155 ceph-mon[35292]: osdmap e18: 3 total, 2 up, 3 in 2024-09-19T06:13:38.634 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:38 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:13:38.634 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:38 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:13:38.634 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:38 smithi155 ceph-mon[35292]: from='osd.2 [v2:172.21.15.53:6818/1988602192,v1:172.21.15.53:6819/1988602192]' entity='osd.2' 2024-09-19T06:13:38.634 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:38 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:13:39.796 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:39 smithi053 ceph-mon[27968]: purged_snaps scrub starts 2024-09-19T06:13:39.796 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:39 smithi053 ceph-mon[27968]: purged_snaps scrub ok 2024-09-19T06:13:39.796 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:39 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:39.796 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:39 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:39.796 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:39 smithi053 ceph-mon[27968]: osd.2 [v2:172.21.15.53:6818/1988602192,v1:172.21.15.53:6819/1988602192] boot 2024-09-19T06:13:39.796 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:39 smithi053 ceph-mon[27968]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T06:13:39.796 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:39 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:13:39.797 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:39 smithi053 ceph-mon[27968]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:13:39.797 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:39 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T06:13:39.899 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:39 smithi155 ceph-mon[35292]: purged_snaps scrub starts 2024-09-19T06:13:39.899 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:39 smithi155 ceph-mon[35292]: purged_snaps scrub ok 2024-09-19T06:13:39.899 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:39 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:39.899 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:39 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:39.900 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:39 smithi155 ceph-mon[35292]: osd.2 [v2:172.21.15.53:6818/1988602192,v1:172.21.15.53:6819/1988602192] boot 2024-09-19T06:13:39.900 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:39 smithi155 ceph-mon[35292]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T06:13:39.900 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:39 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:13:39.900 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:39 smithi155 ceph-mon[35292]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:13:39.900 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:39 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T06:13:40.619 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:13:40.653 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_1 2024-09-19T06:13:40.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:40 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T06:13:40.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:40 smithi155 ceph-mon[35292]: osdmap e20: 3 total, 3 up, 3 in 2024-09-19T06:13:40.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:40 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T06:13:41.044 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:40 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T06:13:41.044 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:40 smithi053 ceph-mon[27968]: osdmap e20: 3 total, 3 up, 3 in 2024-09-19T06:13:41.044 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:40 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T06:13:41.480 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:13:41.736 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:41 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T06:13:41.736 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:41 smithi053 ceph-mon[27968]: osdmap e21: 3 total, 3 up, 3 in 2024-09-19T06:13:41.736 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:41 smithi053 ceph-mon[27968]: pgmap v75: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:13:41.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.926+0000 7f7bd32ba640 1 -- 172.21.15.53:0/1202680160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bcc100860 msgr2=0x7f7bcc100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:41.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.926+0000 7f7bd32ba640 1 --2- 172.21.15.53:0/1202680160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bcc100860 0x7f7bcc100c60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8009880 tx=0x7f7bc802a4a0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.926+0000 7f7bd32ba640 1 -- 172.21.15.53:0/1202680160 shutdown_connections 2024-09-19T06:13:41.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.926+0000 7f7bd32ba640 1 --2- 172.21.15.53:0/1202680160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bcc101a60 0x7f7bcc101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.926+0000 7f7bd32ba640 1 --2- 172.21.15.53:0/1202680160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bcc100860 0x7f7bcc100c60 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.926+0000 7f7bd32ba640 1 -- 172.21.15.53:0/1202680160 >> 172.21.15.53:0/1202680160 conn(0x7f7bcc0fc010 msgr2=0x7f7bcc0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:41.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.927+0000 7f7bd32ba640 1 -- 172.21.15.53:0/1202680160 shutdown_connections 2024-09-19T06:13:41.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.927+0000 7f7bd32ba640 1 -- 172.21.15.53:0/1202680160 wait complete. 2024-09-19T06:13:41.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.928+0000 7f7bd32ba640 1 Processor -- start 2024-09-19T06:13:41.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.928+0000 7f7bd32ba640 1 -- start start 2024-09-19T06:13:41.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.928+0000 7f7bd32ba640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bcc100860 0x7f7bcc06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:41.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.928+0000 7f7bd32ba640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bcc101a60 0x7f7bcc19c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:41.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.928+0000 7f7bd32ba640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc109610 con 0x7f7bcc101a60 2024-09-19T06:13:41.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.928+0000 7f7bd32ba640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc109490 con 0x7f7bcc100860 2024-09-19T06:13:41.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.928+0000 7f7bd1056640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bcc100860 0x7f7bcc06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:41.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.928+0000 7f7bd1056640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bcc100860 0x7f7bcc06a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.53:55338/0 (socket says 172.21.15.53:55338) 2024-09-19T06:13:41.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.928+0000 7f7bd0855640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bcc101a60 0x7f7bcc19c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:41.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.928+0000 7f7bd1056640 1 -- 172.21.15.53:0/1438361959 learned_addr learned my addr 172.21.15.53:0/1438361959 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:13:41.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.929+0000 7f7bd1056640 1 -- 172.21.15.53:0/1438361959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bcc101a60 msgr2=0x7f7bcc19c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:41.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.929+0000 7f7bd1056640 1 --2- 172.21.15.53:0/1438361959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bcc101a60 0x7f7bcc19c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.929+0000 7f7bd1056640 1 -- 172.21.15.53:0/1438361959 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bcc19c6a0 con 0x7f7bcc100860 2024-09-19T06:13:41.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.929+0000 7f7bd1056640 1 --2- 172.21.15.53:0/1438361959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bcc100860 0x7f7bcc06a7d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8006010 tx=0x7f7bc80045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:41.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.929+0000 7f7bc27fc640 1 -- 172.21.15.53:0/1438361959 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7bc803f070 con 0x7f7bcc100860 2024-09-19T06:13:41.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.929+0000 7f7bc27fc640 1 -- 172.21.15.53:0/1438361959 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7bc8004950 con 0x7f7bcc100860 2024-09-19T06:13:41.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.929+0000 7f7bd32ba640 1 -- 172.21.15.53:0/1438361959 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bcc19c930 con 0x7f7bcc100860 2024-09-19T06:13:41.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.929+0000 7f7bc27fc640 1 -- 172.21.15.53:0/1438361959 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7bc8035030 con 0x7f7bcc100860 2024-09-19T06:13:41.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.930+0000 7f7bd32ba640 1 -- 172.21.15.53:0/1438361959 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bcc19ce20 con 0x7f7bcc100860 2024-09-19T06:13:41.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.931+0000 7f7bd32ba640 1 -- 172.21.15.53:0/1438361959 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bcc100c60 con 0x7f7bcc100860 2024-09-19T06:13:41.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.932+0000 7f7bc27fc640 1 -- 172.21.15.53:0/1438361959 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 18) ==== 92495+0+0 (secure 0 0 0) 0x7f7bc8042070 con 0x7f7bcc100860 2024-09-19T06:13:41.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.932+0000 7f7bc27fc640 1 --2- 172.21.15.53:0/1438361959 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f7bb806e230 0x7f7bb80706f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:41.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.933+0000 7f7bc27fc640 1 -- 172.21.15.53:0/1438361959 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 3232+0+0 (secure 0 0 0) 0x7f7bc80ae240 con 0x7f7bcc100860 2024-09-19T06:13:41.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.933+0000 7f7bd0855640 1 --2- 172.21.15.53:0/1438361959 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f7bb806e230 0x7f7bb80706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:41.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.933+0000 7f7bd0855640 1 --2- 172.21.15.53:0/1438361959 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f7bb806e230 0x7f7bb80706f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc06a550 tx=0x7f7bbc005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:41.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:41.938+0000 7f7bc27fc640 1 -- 172.21.15.53:0/1438361959 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7bc8004b00 con 0x7f7bcc100860 2024-09-19T06:13:41.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:41 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T06:13:41.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:41 smithi155 ceph-mon[35292]: osdmap e21: 3 total, 3 up, 3 in 2024-09-19T06:13:41.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:41 smithi155 ceph-mon[35292]: pgmap v75: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:13:42.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:42.170+0000 7f7bd32ba640 1 -- 172.21.15.53:0/1438361959 --> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f7bcc0626f0 con 0x7f7bb806e230 2024-09-19T06:13:42.355 INFO:journalctl@ceph.osd.0.smithi053.stdout:Sep 19 06:13:42 smithi053 sudo[62741]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T06:13:42.355 INFO:journalctl@ceph.osd.0.smithi053.stdout:Sep 19 06:13:42 smithi053 sudo[62741]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:13:42.355 INFO:journalctl@ceph.osd.0.smithi053.stdout:Sep 19 06:13:42 smithi053 sudo[62741]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:13:42.355 INFO:journalctl@ceph.osd.0.smithi053.stdout:Sep 19 06:13:42 smithi053 sudo[62741]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:13:42.770 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:42 smithi053 ceph-mon[27968]: osdmap e22: 3 total, 3 up, 3 in 2024-09-19T06:13:42.770 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:42 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:13:42.770 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:42 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:13:42.770 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:42 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:13:42.770 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:42 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:42.770 INFO:journalctl@ceph.osd.0.smithi053.stdout:Sep 19 06:13:42 smithi053 sudo[63671]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T06:13:42.770 INFO:journalctl@ceph.osd.0.smithi053.stdout:Sep 19 06:13:42 smithi053 sudo[63671]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:13:42.770 INFO:journalctl@ceph.osd.0.smithi053.stdout:Sep 19 06:13:42 smithi053 sudo[63671]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:13:42.771 INFO:journalctl@ceph.osd.0.smithi053.stdout:Sep 19 06:13:42 smithi053 sudo[63671]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:13:42.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:42 smithi155 ceph-mon[35292]: osdmap e22: 3 total, 3 up, 3 in 2024-09-19T06:13:42.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:42 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:13:42.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:42 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:13:42.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:42 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:13:42.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:42 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:43.057 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 19 06:13:42 smithi053 sudo[63755]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T06:13:43.057 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 19 06:13:42 smithi053 sudo[63755]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:13:43.057 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 19 06:13:42 smithi053 sudo[63755]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:13:43.057 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 19 06:13:42 smithi053 sudo[63755]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:13:43.591 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 19 06:13:43 smithi053 sudo[63848]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T06:13:43.591 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 19 06:13:43 smithi053 sudo[63848]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:13:43.591 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 19 06:13:43 smithi053 sudo[63848]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:13:43.591 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 19 06:13:43 smithi053 sudo[63848]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:13:43.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:13:43.641+0000 7f7bc27fc640 1 -- 172.21.15.53:0/1438361959 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f7bc80784d0 con 0x7f7bcc100860 2024-09-19T06:13:43.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:43 smithi053 ceph-mon[27968]: from='client.24145 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:13:43.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:43 smithi053 ceph-mon[27968]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:13:43.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:43 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:43.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:43 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:43.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:43 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:43.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:43 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:43.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:43 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:43.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:43 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:43.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:43 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:13:43.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:43 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:13:43.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:43 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:43.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:43 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:13:43.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:43 smithi155 ceph-mon[35292]: from='client.24145 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:13:43.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:43 smithi155 ceph-mon[35292]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:13:43.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:43 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:43.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:43 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:43.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:43 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:43.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:43 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:43.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:43 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:43.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:43 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:43.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:43 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:13:43.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:43 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:13:43.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:43 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:43.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:43 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:13:44.144 INFO:journalctl@ceph.osd.2.smithi053.stdout:Sep 19 06:13:43 smithi053 sudo[63854]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T06:13:44.145 INFO:journalctl@ceph.osd.2.smithi053.stdout:Sep 19 06:13:43 smithi053 sudo[63854]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:13:44.145 INFO:journalctl@ceph.osd.2.smithi053.stdout:Sep 19 06:13:43 smithi053 sudo[63854]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:13:44.145 INFO:journalctl@ceph.osd.2.smithi053.stdout:Sep 19 06:13:43 smithi053 sudo[63854]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:13:44.421 INFO:journalctl@ceph.osd.2.smithi053.stdout:Sep 19 06:13:44 smithi053 sudo[63906]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T06:13:44.422 INFO:journalctl@ceph.osd.2.smithi053.stdout:Sep 19 06:13:44 smithi053 sudo[63906]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:13:44.422 INFO:journalctl@ceph.osd.2.smithi053.stdout:Sep 19 06:13:44 smithi053 sudo[63906]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:13:44.422 INFO:journalctl@ceph.osd.2.smithi053.stdout:Sep 19 06:13:44 smithi053 sudo[63906]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:13:44.979 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:44 smithi053 sudo[63977]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T06:13:44.979 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:44 smithi053 sudo[63977]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:13:44.979 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:44 smithi053 sudo[63977]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:13:44.979 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:44 smithi053 sudo[63977]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:13:45.546 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:45 smithi155 ceph-mon[35292]: Detected new or changed devices on smithi053 2024-09-19T06:13:45.546 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:45 smithi155 ceph-mon[35292]: Adjusting osd_memory_target on smithi053 to 3948M 2024-09-19T06:13:45.546 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:45 smithi155 ceph-mon[35292]: mgrmap e19: smithi053.trakvg(active, since 2m), standbys: smithi155.piymgv 2024-09-19T06:13:45.546 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:45 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:13:45.547 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:45 smithi155 ceph-mon[35292]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T06:13:45.547 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:45 smithi155 sudo[38566]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T06:13:45.547 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:45 smithi155 sudo[38566]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:13:45.548 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:45 smithi155 sudo[38566]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:13:45.548 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:45 smithi155 sudo[38566]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:13:45.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:45 smithi053 sudo[64021]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T06:13:45.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:45 smithi053 sudo[64021]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:13:45.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:45 smithi053 sudo[64021]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:13:45.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:45 smithi053 sudo[64021]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:13:45.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:45 smithi053 ceph-mon[27968]: Detected new or changed devices on smithi053 2024-09-19T06:13:45.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:45 smithi053 ceph-mon[27968]: Adjusting osd_memory_target on smithi053 to 3948M 2024-09-19T06:13:45.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:45 smithi053 ceph-mon[27968]: mgrmap e19: smithi053.trakvg(active, since 2m), standbys: smithi155.piymgv 2024-09-19T06:13:45.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:45 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:13:45.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:45 smithi053 ceph-mon[27968]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T06:13:45.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:45 smithi155 sudo[38569]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T06:13:45.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:45 smithi155 sudo[38569]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:13:45.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:45 smithi155 sudo[38569]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:13:45.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:45 smithi155 sudo[38569]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:13:46.469 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:46 smithi053 ceph-mon[27968]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:13:46.469 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:46 smithi053 ceph-mon[27968]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T06:13:46.469 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:46 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata", "id": "smithi053"}]: dispatch 2024-09-19T06:13:46.469 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:46 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata", "id": "smithi155"}]: dispatch 2024-09-19T06:13:46.469 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:46 smithi053 ceph-mon[27968]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T06:13:46.469 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:46 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata", "id": "smithi053"}]: dispatch 2024-09-19T06:13:46.470 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:46 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata", "id": "smithi155"}]: dispatch 2024-09-19T06:13:46.470 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:46 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/1690875404' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6bf8cf0-2419-4c2a-b562-9221a117050c"}]: dispatch 2024-09-19T06:13:46.470 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:46 smithi053 ceph-mon[27968]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T06:13:46.470 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:46 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/1690875404' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6bf8cf0-2419-4c2a-b562-9221a117050c"}]': finished 2024-09-19T06:13:46.470 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:46 smithi053 ceph-mon[27968]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T06:13:46.470 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:46 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:13:46.470 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:46 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:46.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:46 smithi155 ceph-mon[35292]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:13:46.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:46 smithi155 ceph-mon[35292]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T06:13:46.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:46 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata", "id": "smithi053"}]: dispatch 2024-09-19T06:13:46.734 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:46 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata", "id": "smithi155"}]: dispatch 2024-09-19T06:13:46.734 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:46 smithi155 ceph-mon[35292]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T06:13:46.734 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:46 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata", "id": "smithi053"}]: dispatch 2024-09-19T06:13:46.734 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:46 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mon metadata", "id": "smithi155"}]: dispatch 2024-09-19T06:13:46.734 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:46 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/1690875404' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6bf8cf0-2419-4c2a-b562-9221a117050c"}]: dispatch 2024-09-19T06:13:46.735 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:46 smithi155 ceph-mon[35292]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T06:13:46.735 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:46 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/1690875404' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6bf8cf0-2419-4c2a-b562-9221a117050c"}]': finished 2024-09-19T06:13:46.735 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:46 smithi155 ceph-mon[35292]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T06:13:46.735 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:46 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:13:46.735 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:46 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:47.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:47 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/130279794' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:13:47.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:47 smithi053 ceph-mon[27968]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:13:47.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:47 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/130279794' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:13:47.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:47 smithi155 ceph-mon[35292]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:13:49.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:49 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:49.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:49 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:49.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:49 smithi155 ceph-mon[35292]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-09-19T06:13:49.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:49 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:13:49.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:49 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:13:49.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:49 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:49.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:49 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:49.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:49 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:49.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:49 smithi053 ceph-mon[27968]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-09-19T06:13:49.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:49 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:13:49.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:49 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:13:49.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:49 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:13:51.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:50 smithi053 ceph-mon[27968]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-19T06:13:51.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:50 smithi155 ceph-mon[35292]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-19T06:13:53.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:52 smithi053 ceph-mon[27968]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-09-19T06:13:53.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:52 smithi155 ceph-mon[35292]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-09-19T06:13:53.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:53 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T06:13:53.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:53 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:13:53.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:53 smithi053 ceph-mon[27968]: Deploying daemon osd.3 on smithi053 2024-09-19T06:13:53.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T06:13:53.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:13:53.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:53 smithi155 ceph-mon[35292]: Deploying daemon osd.3 on smithi053 2024-09-19T06:13:54.922 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:54 smithi053 ceph-mon[27968]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-19T06:13:55.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:54 smithi155 ceph-mon[35292]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-19T06:13:57.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:56 smithi053 ceph-mon[27968]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.6 KiB/s rd, 72 KiB/s wr, 5 op/s 2024-09-19T06:13:57.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:56 smithi155 ceph-mon[35292]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.6 KiB/s rd, 72 KiB/s wr, 5 op/s 2024-09-19T06:13:58.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:58 smithi155 ceph-mon[35292]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-19T06:13:58.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:58 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:13:59.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:58 smithi053 ceph-mon[27968]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-19T06:13:59.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:58 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:14:00.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:14:00.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:00.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:13:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:00.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:14:00.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:00.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:13:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:01.045 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:00 smithi053 ceph-mon[27968]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:14:01.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:00 smithi155 ceph-mon[35292]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:14:02.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:14:02.873+0000 7f7bc27fc640 1 -- 172.21.15.53:0/1438361959 <== mgr.14217 v2:172.21.15.53:6800/3884355752 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7bcc0626f0 con 0x7f7bb806e230 2024-09-19T06:14:02.875 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 3 on host 'smithi053' 2024-09-19T06:14:02.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:14:02.876+0000 7f7bd32ba640 1 -- 172.21.15.53:0/1438361959 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f7bb806e230 msgr2=0x7f7bb80706f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:02.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:14:02.876+0000 7f7bd32ba640 1 --2- 172.21.15.53:0/1438361959 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f7bb806e230 0x7f7bb80706f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc06a550 tx=0x7f7bbc005f90 comp rx=0 tx=0).stop 2024-09-19T06:14:02.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:14:02.876+0000 7f7bd32ba640 1 -- 172.21.15.53:0/1438361959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bcc100860 msgr2=0x7f7bcc06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:02.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:14:02.876+0000 7f7bd32ba640 1 --2- 172.21.15.53:0/1438361959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bcc100860 0x7f7bcc06a7d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8006010 tx=0x7f7bc80045e0 comp rx=0 tx=0).stop 2024-09-19T06:14:02.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:14:02.876+0000 7f7bd32ba640 1 -- 172.21.15.53:0/1438361959 shutdown_connections 2024-09-19T06:14:02.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:14:02.876+0000 7f7bd32ba640 1 --2- 172.21.15.53:0/1438361959 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f7bb806e230 0x7f7bb80706f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:02.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:14:02.876+0000 7f7bd32ba640 1 --2- 172.21.15.53:0/1438361959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bcc101a60 0x7f7bcc19c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:02.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:14:02.876+0000 7f7bd32ba640 1 --2- 172.21.15.53:0/1438361959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bcc100860 0x7f7bcc06a7d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:02.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:14:02.876+0000 7f7bd32ba640 1 -- 172.21.15.53:0/1438361959 >> 172.21.15.53:0/1438361959 conn(0x7f7bcc0fc010 msgr2=0x7f7bcc104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:02.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:14:02.876+0000 7f7bd32ba640 1 -- 172.21.15.53:0/1438361959 shutdown_connections 2024-09-19T06:14:02.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:14:02.876+0000 7f7bd32ba640 1 -- 172.21.15.53:0/1438361959 wait complete. 2024-09-19T06:14:03.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:02 smithi053 ceph-mon[27968]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:14:03.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:02 smithi155 ceph-mon[35292]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:14:03.513 DEBUG:teuthology.orchestra.run.smithi053:osd.3> sudo journalctl -f -n 0 -u ceph-9f2876d0-764d-11ef-bceb-c7b262605968@osd.3.service 2024-09-19T06:14:03.516 INFO:tasks.cephadm:Deploying osd.4 on smithi155 with /dev/vg_nvme/lv_4... 2024-09-19T06:14:03.516 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T06:14:03.774 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi155/config 2024-09-19T06:14:04.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:03 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:04.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:03 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:04.105 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 19 06:14:04 smithi053 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-3[66229]: 2024-09-19T06:14:04.010+0000 7f674ed93640 -1 osd.3 0 log_to_monitors true 2024-09-19T06:14:04.147 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:03 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:04.148 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:03 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:05.151 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:04 smithi155 ceph-mon[35292]: from='osd.3 [v2:172.21.15.53:6826/2604012854,v1:172.21.15.53:6827/2604012854]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T06:14:05.151 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:04 smithi155 ceph-mon[35292]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:14:05.247 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:04 smithi053 ceph-mon[27968]: from='osd.3 [v2:172.21.15.53:6826/2604012854,v1:172.21.15.53:6827/2604012854]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T06:14:05.247 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:04 smithi053 ceph-mon[27968]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:14:06.140 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:05 smithi053 ceph-mon[27968]: from='osd.3 [v2:172.21.15.53:6826/2604012854,v1:172.21.15.53:6827/2604012854]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T06:14:06.140 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:05 smithi053 ceph-mon[27968]: osdmap e24: 4 total, 3 up, 4 in 2024-09-19T06:14:06.140 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:05 smithi053 ceph-mon[27968]: from='osd.3 [v2:172.21.15.53:6826/2604012854,v1:172.21.15.53:6827/2604012854]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-19T06:14:06.140 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:05 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:14:06.140 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:05 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:06.140 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:05 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:06.200 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:05 smithi155 ceph-mon[35292]: from='osd.3 [v2:172.21.15.53:6826/2604012854,v1:172.21.15.53:6827/2604012854]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T06:14:06.200 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:05 smithi155 ceph-mon[35292]: osdmap e24: 4 total, 3 up, 4 in 2024-09-19T06:14:06.200 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:05 smithi155 ceph-mon[35292]: from='osd.3 [v2:172.21.15.53:6826/2604012854,v1:172.21.15.53:6827/2604012854]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-19T06:14:06.200 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:05 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:14:06.200 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:05 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:06.200 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:05 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:06.474 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 19 06:14:06 smithi053 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-3[66229]: 2024-09-19T06:14:06.138+0000 7f674ad16640 -1 osd.3 0 waiting for initial osdmap 2024-09-19T06:14:06.474 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 19 06:14:06 smithi053 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-3[66229]: 2024-09-19T06:14:06.141+0000 7f6742b36640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:14:06.513 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:14:06.541 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi155:vg_nvme/lv_4 2024-09-19T06:14:06.791 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi155/config 2024-09-19T06:14:07.146 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:06 smithi155 ceph-mon[35292]: from='osd.3 [v2:172.21.15.53:6826/2604012854,v1:172.21.15.53:6827/2604012854]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-19T06:14:07.146 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:06 smithi155 ceph-mon[35292]: osdmap e25: 4 total, 3 up, 4 in 2024-09-19T06:14:07.146 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:06 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:14:07.146 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:06 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:14:07.146 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:06 smithi155 ceph-mon[35292]: from='osd.3 [v2:172.21.15.53:6826/2604012854,v1:172.21.15.53:6827/2604012854]' entity='osd.3' 2024-09-19T06:14:07.146 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:06 smithi155 ceph-mon[35292]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:14:07.146 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:06 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:14:07.183 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:06 smithi053 ceph-mon[27968]: from='osd.3 [v2:172.21.15.53:6826/2604012854,v1:172.21.15.53:6827/2604012854]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-19T06:14:07.183 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:06 smithi053 ceph-mon[27968]: osdmap e25: 4 total, 3 up, 4 in 2024-09-19T06:14:07.183 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:06 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:14:07.183 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:06 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:14:07.183 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:06 smithi053 ceph-mon[27968]: from='osd.3 [v2:172.21.15.53:6826/2604012854,v1:172.21.15.53:6827/2604012854]' entity='osd.3' 2024-09-19T06:14:07.183 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:06 smithi053 ceph-mon[27968]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:14:07.183 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:06 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:14:07.232 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.230+0000 7f9d90776640 1 -- 172.21.15.155:0/2673874346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d88101a60 msgr2=0x7f9d88101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:07.232 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.230+0000 7f9d90776640 1 --2- 172.21.15.155:0/2673874346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d88101a60 0x7f9d88101ee0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9d78009880 tx=0x7f9d7802a4a0 comp rx=0 tx=0).stop 2024-09-19T06:14:07.232 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.230+0000 7f9d90776640 1 -- 172.21.15.155:0/2673874346 shutdown_connections 2024-09-19T06:14:07.232 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.230+0000 7f9d90776640 1 --2- 172.21.15.155:0/2673874346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d88101a60 0x7f9d88101ee0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:07.232 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.230+0000 7f9d90776640 1 --2- 172.21.15.155:0/2673874346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d88100860 0x7f9d88100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:07.232 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.230+0000 7f9d90776640 1 -- 172.21.15.155:0/2673874346 >> 172.21.15.155:0/2673874346 conn(0x7f9d880fc010 msgr2=0x7f9d880fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:07.232 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.230+0000 7f9d90776640 1 -- 172.21.15.155:0/2673874346 shutdown_connections 2024-09-19T06:14:07.232 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.230+0000 7f9d90776640 1 -- 172.21.15.155:0/2673874346 wait complete. 2024-09-19T06:14:07.232 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.231+0000 7f9d90776640 1 Processor -- start 2024-09-19T06:14:07.233 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.231+0000 7f9d90776640 1 -- start start 2024-09-19T06:14:07.234 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.231+0000 7f9d90776640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d88100860 0x7f9d8806a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:07.234 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.231+0000 7f9d90776640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d88101a60 0x7f9d8819c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:07.234 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.231+0000 7f9d90776640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9d881095f0 con 0x7f9d88101a60 2024-09-19T06:14:07.234 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.231+0000 7f9d90776640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9d88109470 con 0x7f9d88100860 2024-09-19T06:14:07.234 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.231+0000 7f9d8e512640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d88100860 0x7f9d8806a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:07.234 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.231+0000 7f9d8e512640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d88100860 0x7f9d8806a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:40512/0 (socket says 172.21.15.155:40512) 2024-09-19T06:14:07.235 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.231+0000 7f9d8e512640 1 -- 172.21.15.155:0/2544192224 learned_addr learned my addr 172.21.15.155:0/2544192224 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-19T06:14:07.235 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.231+0000 7f9d8dd11640 1 --2- 172.21.15.155:0/2544192224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d88101a60 0x7f9d8819c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:07.235 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.231+0000 7f9d8e512640 1 -- 172.21.15.155:0/2544192224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d88101a60 msgr2=0x7f9d8819c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:07.235 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.231+0000 7f9d8e512640 1 --2- 172.21.15.155:0/2544192224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d88101a60 0x7f9d8819c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:07.235 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.231+0000 7f9d8e512640 1 -- 172.21.15.155:0/2544192224 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8819c690 con 0x7f9d88100860 2024-09-19T06:14:07.235 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.231+0000 7f9d8e512640 1 --2- 172.21.15.155:0/2544192224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d88100860 0x7f9d8806a790 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400c8f0 tx=0x7f9d8400cdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:07.235 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.232+0000 7f9d7f7fe640 1 -- 172.21.15.155:0/2544192224 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9d840079b0 con 0x7f9d88100860 2024-09-19T06:14:07.235 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.232+0000 7f9d7f7fe640 1 -- 172.21.15.155:0/2544192224 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d8400f450 con 0x7f9d88100860 2024-09-19T06:14:07.235 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.232+0000 7f9d90776640 1 -- 172.21.15.155:0/2544192224 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d8819c980 con 0x7f9d88100860 2024-09-19T06:14:07.235 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.232+0000 7f9d7f7fe640 1 -- 172.21.15.155:0/2544192224 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9d840185d0 con 0x7f9d88100860 2024-09-19T06:14:07.235 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.232+0000 7f9d90776640 1 -- 172.21.15.155:0/2544192224 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d8819cf90 con 0x7f9d88100860 2024-09-19T06:14:07.235 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.233+0000 7f9d90776640 1 -- 172.21.15.155:0/2544192224 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d88100c60 con 0x7f9d88100860 2024-09-19T06:14:07.236 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.233+0000 7f9d7f7fe640 1 -- 172.21.15.155:0/2544192224 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f9d8400f600 con 0x7f9d88100860 2024-09-19T06:14:07.236 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.234+0000 7f9d7f7fe640 1 --2- 172.21.15.155:0/2544192224 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f9d7406e2b0 0x7f9d74070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:07.236 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.234+0000 7f9d7f7fe640 1 -- 172.21.15.155:0/2544192224 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 3664+0+0 (secure 0 0 0) 0x7f9d8408dcd0 con 0x7f9d88100860 2024-09-19T06:14:07.236 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.234+0000 7f9d8dd11640 1 --2- 172.21.15.155:0/2544192224 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f9d7406e2b0 0x7f9d74070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:07.236 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.234+0000 7f9d8dd11640 1 --2- 172.21.15.155:0/2544192224 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f9d7406e2b0 0x7f9d74070770 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9d8806a510 tx=0x7f9d78004e20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:07.240 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.239+0000 7f9d7f7fe640 1 -- 172.21.15.155:0/2544192224 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9d84014030 con 0x7f9d88100860 2024-09-19T06:14:07.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:07.420+0000 7f9d90776640 1 -- 172.21.15.155:0/2544192224 --> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f9d881060b0 con 0x7f9d7406e2b0 2024-09-19T06:14:08.073 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:07 smithi155 ceph-mon[35292]: purged_snaps scrub starts 2024-09-19T06:14:08.074 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:07 smithi155 ceph-mon[35292]: purged_snaps scrub ok 2024-09-19T06:14:08.074 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:07 smithi155 ceph-mon[35292]: osd.3 [v2:172.21.15.53:6826/2604012854,v1:172.21.15.53:6827/2604012854] boot 2024-09-19T06:14:08.074 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:07 smithi155 ceph-mon[35292]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T06:14:08.074 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:07 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:14:08.074 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:07 smithi155 ceph-mon[35292]: from='client.24161 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:14:08.074 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:07 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:14:08.074 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:07 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:14:08.074 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:07 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:14:08.151 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:07 smithi053 ceph-mon[27968]: purged_snaps scrub starts 2024-09-19T06:14:08.152 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:07 smithi053 ceph-mon[27968]: purged_snaps scrub ok 2024-09-19T06:14:08.152 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:07 smithi053 ceph-mon[27968]: osd.3 [v2:172.21.15.53:6826/2604012854,v1:172.21.15.53:6827/2604012854] boot 2024-09-19T06:14:08.152 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:07 smithi053 ceph-mon[27968]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T06:14:08.152 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:07 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:14:08.152 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:07 smithi053 ceph-mon[27968]: from='client.24161 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:14:08.152 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:07 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:14:08.152 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:07 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:14:08.152 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:07 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:14:09.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:08 smithi053 ceph-mon[27968]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:14:09.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:08 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:09.221 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:08 smithi155 ceph-mon[35292]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:14:09.221 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:08 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:10.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:09 smithi155 ceph-mon[35292]: Detected new or changed devices on smithi053 2024-09-19T06:14:10.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:09 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:10.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:09 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:14:10.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:09 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:14:10.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:09 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:14:10.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:09 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:14:10.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:09 smithi155 ceph-mon[35292]: Adjusting osd_memory_target on smithi053 to 2961M 2024-09-19T06:14:10.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:09 smithi155 ceph-mon[35292]: osdmap e27: 4 total, 4 up, 4 in 2024-09-19T06:14:10.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:09 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:10.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:09 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:14:10.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:09 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:14:10.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:09 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:10.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:09 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:14:10.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:09 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:14:10.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:09 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:14:10.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:09 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:10.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:09 smithi155 ceph-mon[35292]: from='client.? 172.21.15.155:0/3741054306' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7adedf0a-6fbc-42bc-8996-ba482b638929"}]: dispatch 2024-09-19T06:14:10.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:09 smithi155 ceph-mon[35292]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7adedf0a-6fbc-42bc-8996-ba482b638929"}]: dispatch 2024-09-19T06:14:10.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:09 smithi155 ceph-mon[35292]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7adedf0a-6fbc-42bc-8996-ba482b638929"}]': finished 2024-09-19T06:14:10.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:09 smithi155 ceph-mon[35292]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T06:14:10.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:09 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:14:10.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:09 smithi053 ceph-mon[27968]: Detected new or changed devices on smithi053 2024-09-19T06:14:10.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:09 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:10.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:09 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:14:10.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:09 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:14:10.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:09 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:14:10.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:09 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:14:10.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:09 smithi053 ceph-mon[27968]: Adjusting osd_memory_target on smithi053 to 2961M 2024-09-19T06:14:10.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:09 smithi053 ceph-mon[27968]: osdmap e27: 4 total, 4 up, 4 in 2024-09-19T06:14:10.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:09 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:10.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:09 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:14:10.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:09 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:14:10.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:09 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:10.357 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:09 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:14:10.357 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:09 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:14:10.357 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:09 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:14:10.357 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:09 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:10.357 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:09 smithi053 ceph-mon[27968]: from='client.? 172.21.15.155:0/3741054306' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7adedf0a-6fbc-42bc-8996-ba482b638929"}]: dispatch 2024-09-19T06:14:10.357 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:09 smithi053 ceph-mon[27968]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7adedf0a-6fbc-42bc-8996-ba482b638929"}]: dispatch 2024-09-19T06:14:10.357 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:09 smithi053 ceph-mon[27968]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7adedf0a-6fbc-42bc-8996-ba482b638929"}]': finished 2024-09-19T06:14:10.357 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:09 smithi053 ceph-mon[27968]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T06:14:10.357 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:09 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:14:11.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:10 smithi053 ceph-mon[27968]: from='client.? 172.21.15.155:0/3795897968' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:14:11.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:10 smithi053 ceph-mon[27968]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:14:11.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:10 smithi155 ceph-mon[35292]: from='client.? 172.21.15.155:0/3795897968' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:14:11.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:10 smithi155 ceph-mon[35292]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:14:13.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:12 smithi053 ceph-mon[27968]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:14:13.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:12 smithi155 ceph-mon[35292]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:14:14.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:13 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:14:14.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:13 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:14:14.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:14 smithi155 ceph-mon[35292]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:14:15.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:14 smithi053 ceph-mon[27968]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:14:17.080 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:16 smithi155 ceph-mon[35292]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:14:17.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:16 smithi053 ceph-mon[27968]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:14:18.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:17 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T06:14:18.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:17 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:14:18.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:17 smithi053 ceph-mon[27968]: Deploying daemon osd.4 on smithi155 2024-09-19T06:14:18.115 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:17 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T06:14:18.115 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:17 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:14:18.116 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:17 smithi155 ceph-mon[35292]: Deploying daemon osd.4 on smithi155 2024-09-19T06:14:19.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:18 smithi053 ceph-mon[27968]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:14:19.113 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:18 smithi155 ceph-mon[35292]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:14:21.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:20 smithi053 ceph-mon[27968]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:14:21.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:20 smithi155 ceph-mon[35292]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:14:22.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:22 smithi155 ceph-mon[35292]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:14:23.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:22 smithi053 ceph-mon[27968]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:14:23.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:23 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:14:23.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:23 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:23.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:23 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:24.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:23 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:14:24.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:23 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:24.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:23 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:25.083 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:24 smithi155 ceph-mon[35292]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:14:25.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:24 smithi053 ceph-mon[27968]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:14:26.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:26 smithi155 ceph-mon[35292]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:14:27.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:26 smithi053 ceph-mon[27968]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:14:27.525 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:27.524+0000 7f9d7f7fe640 1 -- 172.21.15.155:0/2544192224 <== mgr.14217 v2:172.21.15.53:6800/3884355752 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9d881060b0 con 0x7f9d7406e2b0 2024-09-19T06:14:27.525 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 4 on host 'smithi155' 2024-09-19T06:14:27.528 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:27.526+0000 7f9d90776640 1 -- 172.21.15.155:0/2544192224 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f9d7406e2b0 msgr2=0x7f9d74070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:27.528 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:27.526+0000 7f9d90776640 1 --2- 172.21.15.155:0/2544192224 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f9d7406e2b0 0x7f9d74070770 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9d8806a510 tx=0x7f9d78004e20 comp rx=0 tx=0).stop 2024-09-19T06:14:27.528 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:27.527+0000 7f9d90776640 1 -- 172.21.15.155:0/2544192224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d88100860 msgr2=0x7f9d8806a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:27.528 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:27.527+0000 7f9d90776640 1 --2- 172.21.15.155:0/2544192224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d88100860 0x7f9d8806a790 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400c8f0 tx=0x7f9d8400cdb0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.529 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:27.527+0000 7f9d90776640 1 -- 172.21.15.155:0/2544192224 shutdown_connections 2024-09-19T06:14:27.529 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:27.527+0000 7f9d90776640 1 --2- 172.21.15.155:0/2544192224 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f9d7406e2b0 0x7f9d74070770 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.529 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:27.527+0000 7f9d90776640 1 --2- 172.21.15.155:0/2544192224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d88101a60 0x7f9d8819c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.529 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:27.527+0000 7f9d90776640 1 --2- 172.21.15.155:0/2544192224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d88100860 0x7f9d8806a790 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.529 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:27.527+0000 7f9d90776640 1 -- 172.21.15.155:0/2544192224 >> 172.21.15.155:0/2544192224 conn(0x7f9d880fc010 msgr2=0x7f9d880fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:27.529 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:27.527+0000 7f9d90776640 1 -- 172.21.15.155:0/2544192224 shutdown_connections 2024-09-19T06:14:27.529 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:27.527+0000 7f9d90776640 1 -- 172.21.15.155:0/2544192224 wait complete. 2024-09-19T06:14:28.146 DEBUG:teuthology.orchestra.run.smithi155:osd.4> sudo journalctl -f -n 0 -u ceph-9f2876d0-764d-11ef-bceb-c7b262605968@osd.4.service 2024-09-19T06:14:28.148 INFO:tasks.cephadm:Deploying osd.5 on smithi155 with /dev/vg_nvme/lv_3... 2024-09-19T06:14:28.149 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T06:14:28.443 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi155/config 2024-09-19T06:14:28.779 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:28 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:28.779 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:28 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:28.779 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:28 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:28.779 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:28 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:28.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:28 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:28.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:28 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:28.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:28 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:28.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:28 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:29.215 INFO:journalctl@ceph.osd.4.smithi155.stdout:Sep 19 06:14:28 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-4[40915]: 2024-09-19T06:14:28.834+0000 7f7d5bf2c640 -1 osd.4 0 log_to_monitors true 2024-09-19T06:14:29.765 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:29 smithi155 ceph-mon[35292]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:14:29.766 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:29 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:14:29.766 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:29 smithi155 ceph-mon[35292]: from='osd.4 [v2:172.21.15.155:6800/2046029108,v1:172.21.15.155:6801/2046029108]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T06:14:29.766 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:29 smithi155 ceph-mon[35292]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T06:14:29.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:29 smithi053 ceph-mon[27968]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:14:29.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:14:29.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:29 smithi053 ceph-mon[27968]: from='osd.4 [v2:172.21.15.155:6800/2046029108,v1:172.21.15.155:6801/2046029108]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T06:14:29.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:29 smithi053 ceph-mon[27968]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T06:14:30.784 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:30 smithi155 ceph-mon[35292]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T06:14:30.784 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:30 smithi155 ceph-mon[35292]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T06:14:30.784 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:30 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:14:30.784 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:30 smithi155 ceph-mon[35292]: from='osd.4 [v2:172.21.15.155:6800/2046029108,v1:172.21.15.155:6801/2046029108]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-19T06:14:30.784 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:30 smithi155 ceph-mon[35292]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-19T06:14:30.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:30 smithi053 ceph-mon[27968]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T06:14:30.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:30 smithi053 ceph-mon[27968]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T06:14:30.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:30 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:14:30.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:30 smithi053 ceph-mon[27968]: from='osd.4 [v2:172.21.15.155:6800/2046029108,v1:172.21.15.155:6801/2046029108]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-19T06:14:30.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:30 smithi053 ceph-mon[27968]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-19T06:14:31.141 INFO:journalctl@ceph.osd.4.smithi155.stdout:Sep 19 06:14:30 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-4[40915]: 2024-09-19T06:14:30.973+0000 7f7d57eaf640 -1 osd.4 0 waiting for initial osdmap 2024-09-19T06:14:31.141 INFO:journalctl@ceph.osd.4.smithi155.stdout:Sep 19 06:14:30 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-4[40915]: 2024-09-19T06:14:30.979+0000 7f7d4fccf640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:14:31.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:31 smithi053 ceph-mon[27968]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-09-19T06:14:31.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:31 smithi053 ceph-mon[27968]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T06:14:31.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:31 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:14:31.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:31 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:14:31.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:31 smithi053 ceph-mon[27968]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:14:31.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:31 smithi053 ceph-mon[27968]: from='osd.4 ' entity='osd.4' 2024-09-19T06:14:31.870 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:31 smithi155 ceph-mon[35292]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-09-19T06:14:31.870 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:31 smithi155 ceph-mon[35292]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T06:14:31.870 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:31 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:14:31.870 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:31 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:14:31.870 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:31 smithi155 ceph-mon[35292]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:14:31.870 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:31 smithi155 ceph-mon[35292]: from='osd.4 ' entity='osd.4' 2024-09-19T06:14:31.872 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:14:31.899 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi155:vg_nvme/lv_3 2024-09-19T06:14:32.433 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi155/config 2024-09-19T06:14:32.843 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:32 smithi155 ceph-mon[35292]: purged_snaps scrub starts 2024-09-19T06:14:32.844 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:32 smithi155 ceph-mon[35292]: purged_snaps scrub ok 2024-09-19T06:14:32.844 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:32 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:14:32.844 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:32 smithi155 ceph-mon[35292]: osd.4 [v2:172.21.15.155:6800/2046029108,v1:172.21.15.155:6801/2046029108] boot 2024-09-19T06:14:32.844 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:32 smithi155 ceph-mon[35292]: osdmap e31: 5 total, 5 up, 5 in 2024-09-19T06:14:32.844 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:32 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:14:32.844 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:32 smithi155 ceph-mon[35292]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T06:14:32.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:32 smithi053 ceph-mon[27968]: purged_snaps scrub starts 2024-09-19T06:14:32.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:32 smithi053 ceph-mon[27968]: purged_snaps scrub ok 2024-09-19T06:14:32.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:32 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:14:32.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:32 smithi053 ceph-mon[27968]: osd.4 [v2:172.21.15.155:6800/2046029108,v1:172.21.15.155:6801/2046029108] boot 2024-09-19T06:14:32.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:32 smithi053 ceph-mon[27968]: osdmap e31: 5 total, 5 up, 5 in 2024-09-19T06:14:32.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:32 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:14:32.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:32 smithi053 ceph-mon[27968]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T06:14:32.928 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.926+0000 7f821d121640 1 -- 172.21.15.155:0/1286298029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8218100860 msgr2=0x7f8218100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:32.928 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.926+0000 7f821d121640 1 --2- 172.21.15.155:0/1286298029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8218100860 0x7f8218100c60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8208009880 tx=0x7f820802a4a0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.928 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.926+0000 7f821d121640 1 -- 172.21.15.155:0/1286298029 shutdown_connections 2024-09-19T06:14:32.928 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.926+0000 7f821d121640 1 --2- 172.21.15.155:0/1286298029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8218101a60 0x7f8218101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.928 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.926+0000 7f821d121640 1 --2- 172.21.15.155:0/1286298029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8218100860 0x7f8218100c60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.928 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.926+0000 7f821d121640 1 -- 172.21.15.155:0/1286298029 >> 172.21.15.155:0/1286298029 conn(0x7f82180fc010 msgr2=0x7f82180fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:32.928 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.926+0000 7f821d121640 1 -- 172.21.15.155:0/1286298029 shutdown_connections 2024-09-19T06:14:32.928 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.926+0000 7f821d121640 1 -- 172.21.15.155:0/1286298029 wait complete. 2024-09-19T06:14:32.930 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.927+0000 7f821d121640 1 Processor -- start 2024-09-19T06:14:32.930 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.927+0000 7f821d121640 1 -- start start 2024-09-19T06:14:32.930 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.927+0000 7f821d121640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8218100860 0x7f8218197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:32.930 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.927+0000 7f821d121640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8218101a60 0x7f82181979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:32.930 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.927+0000 7f821d121640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8218109610 con 0x7f8218100860 2024-09-19T06:14:32.930 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.927+0000 7f821d121640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8218109490 con 0x7f8218101a60 2024-09-19T06:14:32.930 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.927+0000 7f821659c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8218101a60 0x7f82181979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:32.931 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.927+0000 7f821659c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8218101a60 0x7f82181979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:46980/0 (socket says 172.21.15.155:46980) 2024-09-19T06:14:32.931 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.927+0000 7f821659c640 1 -- 172.21.15.155:0/2582230771 learned_addr learned my addr 172.21.15.155:0/2582230771 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-19T06:14:32.931 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.928+0000 7f8216d9d640 1 --2- 172.21.15.155:0/2582230771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8218100860 0x7f8218197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:32.931 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.928+0000 7f821659c640 1 -- 172.21.15.155:0/2582230771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8218100860 msgr2=0x7f8218197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:32.931 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.928+0000 7f821659c640 1 --2- 172.21.15.155:0/2582230771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8218100860 0x7f8218197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.931 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.928+0000 7f821659c640 1 -- 172.21.15.155:0/2582230771 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8218197f90 con 0x7f8218101a60 2024-09-19T06:14:32.931 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.928+0000 7f821659c640 1 --2- 172.21.15.155:0/2582230771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8218101a60 0x7f82181979a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f820c00d6f0 tx=0x7f820c00dbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:32.931 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.928+0000 7f81fffff640 1 -- 172.21.15.155:0/2582230771 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f820c017070 con 0x7f8218101a60 2024-09-19T06:14:32.931 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.928+0000 7f81fffff640 1 -- 172.21.15.155:0/2582230771 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f820c00de00 con 0x7f8218101a60 2024-09-19T06:14:32.931 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.928+0000 7f821d121640 1 -- 172.21.15.155:0/2582230771 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f821819ca00 con 0x7f8218101a60 2024-09-19T06:14:32.931 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.928+0000 7f81fffff640 1 -- 172.21.15.155:0/2582230771 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f820c004aa0 con 0x7f8218101a60 2024-09-19T06:14:32.931 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.928+0000 7f821d121640 1 -- 172.21.15.155:0/2582230771 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f821819cfe0 con 0x7f8218101a60 2024-09-19T06:14:32.932 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.929+0000 7f821d121640 1 -- 172.21.15.155:0/2582230771 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8218100c60 con 0x7f8218101a60 2024-09-19T06:14:32.932 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.930+0000 7f81fffff640 1 -- 172.21.15.155:0/2582230771 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f820c004d50 con 0x7f8218101a60 2024-09-19T06:14:32.932 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.930+0000 7f81fffff640 1 --2- 172.21.15.155:0/2582230771 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f820006e2b0 0x7f8200070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:32.932 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.931+0000 7f81fffff640 1 -- 172.21.15.155:0/2582230771 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4235+0+0 (secure 0 0 0) 0x7f820c08d940 con 0x7f8218101a60 2024-09-19T06:14:32.933 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.931+0000 7f8216d9d640 1 --2- 172.21.15.155:0/2582230771 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f820006e2b0 0x7f8200070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:32.933 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.931+0000 7f8216d9d640 1 --2- 172.21.15.155:0/2582230771 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f820006e2b0 0x7f8200070770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8208000c00 tx=0x7f8208009630 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:32.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:32.935+0000 7f81fffff640 1 -- 172.21.15.155:0/2582230771 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f820c05b6e0 con 0x7f8218101a60 2024-09-19T06:14:33.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:33.115+0000 7f821d121640 1 -- 172.21.15.155:0/2582230771 --> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f821802dba0 con 0x7f820006e2b0 2024-09-19T06:14:33.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:33 smithi053 ceph-mon[27968]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:14:33.902 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:33 smithi053 ceph-mon[27968]: osdmap e33: 5 total, 5 up, 5 in 2024-09-19T06:14:33.902 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:33 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:14:33.902 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:33 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:14:33.902 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:33 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:14:33.903 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:33 smithi155 ceph-mon[35292]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:14:33.903 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:33 smithi155 ceph-mon[35292]: osdmap e33: 5 total, 5 up, 5 in 2024-09-19T06:14:33.903 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:33 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:14:33.903 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:33 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:14:33.903 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:33 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:14:34.920 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:34 smithi155 ceph-mon[35292]: from='client.24179 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:14:34.920 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:34 smithi155 ceph-mon[35292]: Detected new or changed devices on smithi155 2024-09-19T06:14:34.920 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:34 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:34.920 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:34 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:34.921 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:34 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:14:34.921 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:34 smithi155 ceph-mon[35292]: Adjusting osd_memory_target on smithi155 to 14912M 2024-09-19T06:14:34.921 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:34 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:34.921 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:34 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:14:34.921 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:34 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:14:34.921 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:34 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:34.921 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:34 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:14:34.921 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:34 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:14:34.922 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:34 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:14:34.922 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:34 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:34.922 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:34 smithi155 ceph-mon[35292]: osdmap e34: 5 total, 5 up, 5 in 2024-09-19T06:14:35.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:34 smithi053 ceph-mon[27968]: from='client.24179 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:14:35.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:34 smithi053 ceph-mon[27968]: Detected new or changed devices on smithi155 2024-09-19T06:14:35.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:34 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:35.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:34 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:35.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:34 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:14:35.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:34 smithi053 ceph-mon[27968]: Adjusting osd_memory_target on smithi155 to 14912M 2024-09-19T06:14:35.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:34 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:35.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:34 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:14:35.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:34 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:14:35.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:34 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:35.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:34 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:14:35.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:34 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:14:35.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:34 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:14:35.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:34 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:35.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:34 smithi053 ceph-mon[27968]: osdmap e34: 5 total, 5 up, 5 in 2024-09-19T06:14:35.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:35 smithi053 ceph-mon[27968]: pgmap v115: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T06:14:35.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:35 smithi053 ceph-mon[27968]: osdmap e35: 5 total, 5 up, 5 in 2024-09-19T06:14:35.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:35 smithi155 ceph-mon[35292]: pgmap v115: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T06:14:35.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:35 smithi155 ceph-mon[35292]: osdmap e35: 5 total, 5 up, 5 in 2024-09-19T06:14:36.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:36 smithi155 ceph-mon[35292]: from='client.? 172.21.15.155:0/705386736' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c664e57c-20b5-46d4-8830-70780a91d6dc"}]: dispatch 2024-09-19T06:14:36.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:36 smithi155 ceph-mon[35292]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c664e57c-20b5-46d4-8830-70780a91d6dc"}]: dispatch 2024-09-19T06:14:36.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:36 smithi155 ceph-mon[35292]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c664e57c-20b5-46d4-8830-70780a91d6dc"}]': finished 2024-09-19T06:14:36.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:36 smithi155 ceph-mon[35292]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T06:14:36.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:36 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:14:36.734 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:36 smithi155 ceph-mon[35292]: from='client.? 172.21.15.155:0/376764595' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:14:37.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:36 smithi053 ceph-mon[27968]: from='client.? 172.21.15.155:0/705386736' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c664e57c-20b5-46d4-8830-70780a91d6dc"}]: dispatch 2024-09-19T06:14:37.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:36 smithi053 ceph-mon[27968]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c664e57c-20b5-46d4-8830-70780a91d6dc"}]: dispatch 2024-09-19T06:14:37.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:36 smithi053 ceph-mon[27968]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c664e57c-20b5-46d4-8830-70780a91d6dc"}]': finished 2024-09-19T06:14:37.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:36 smithi053 ceph-mon[27968]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T06:14:37.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:36 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:14:37.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:36 smithi053 ceph-mon[27968]: from='client.? 172.21.15.155:0/376764595' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:14:37.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:37 smithi155 ceph-mon[35292]: pgmap v118: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T06:14:38.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:37 smithi053 ceph-mon[27968]: pgmap v118: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T06:14:38.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:38 smithi155 ceph-mon[35292]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 152 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T06:14:39.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:38 smithi053 ceph-mon[27968]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 152 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T06:14:40.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:40 smithi155 ceph-mon[35292]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:14:41.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:40 smithi053 ceph-mon[27968]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:14:42.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:42 smithi155 ceph-mon[35292]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 152 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:14:43.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:42 smithi053 ceph-mon[27968]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 152 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:14:43.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:43 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T06:14:43.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:43 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:14:43.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:43 smithi155 ceph-mon[35292]: Deploying daemon osd.5 on smithi155 2024-09-19T06:14:43.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:43 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:14:44.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:43 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T06:14:44.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:43 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:14:44.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:43 smithi053 ceph-mon[27968]: Deploying daemon osd.5 on smithi155 2024-09-19T06:14:44.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:43 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:14:45.098 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:44 smithi155 ceph-mon[35292]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 152 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:14:45.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:44 smithi053 ceph-mon[27968]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 152 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:14:47.030 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:46 smithi155 ceph-mon[35292]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 152 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:14:47.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:46 smithi053 ceph-mon[27968]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 152 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:14:48.918 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:48 smithi155 ceph-mon[35292]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 152 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:14:49.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:48 smithi053 ceph-mon[27968]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 152 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:14:50.100 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:49 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:14:50.101 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:49 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:50.101 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:49 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:50.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:49 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:14:50.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:49 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:50.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:49 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:51.097 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:50 smithi155 ceph-mon[35292]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 152 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:14:51.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:50 smithi053 ceph-mon[27968]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 152 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:14:53.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:52 smithi053 ceph-mon[27968]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 152 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:14:53.140 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:52 smithi155 ceph-mon[35292]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 152 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:14:53.454 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:53.452+0000 7f81fffff640 1 -- 172.21.15.155:0/2582230771 <== mgr.14217 v2:172.21.15.53:6800/3884355752 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f821802dba0 con 0x7f820006e2b0 2024-09-19T06:14:53.454 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 5 on host 'smithi155' 2024-09-19T06:14:53.457 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:53.455+0000 7f821d121640 1 -- 172.21.15.155:0/2582230771 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f820006e2b0 msgr2=0x7f8200070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:53.457 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:53.455+0000 7f821d121640 1 --2- 172.21.15.155:0/2582230771 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f820006e2b0 0x7f8200070770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8208000c00 tx=0x7f8208009630 comp rx=0 tx=0).stop 2024-09-19T06:14:53.457 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:53.455+0000 7f821d121640 1 -- 172.21.15.155:0/2582230771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8218101a60 msgr2=0x7f82181979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:53.457 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:53.455+0000 7f821d121640 1 --2- 172.21.15.155:0/2582230771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8218101a60 0x7f82181979a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f820c00d6f0 tx=0x7f820c00dbb0 comp rx=0 tx=0).stop 2024-09-19T06:14:53.457 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:53.455+0000 7f821d121640 1 -- 172.21.15.155:0/2582230771 shutdown_connections 2024-09-19T06:14:53.457 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:53.455+0000 7f821d121640 1 --2- 172.21.15.155:0/2582230771 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f820006e2b0 0x7f8200070770 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:53.457 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:53.455+0000 7f821d121640 1 --2- 172.21.15.155:0/2582230771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8218101a60 0x7f82181979a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:53.458 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:53.455+0000 7f821d121640 1 --2- 172.21.15.155:0/2582230771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8218100860 0x7f8218197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:53.458 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:53.455+0000 7f821d121640 1 -- 172.21.15.155:0/2582230771 >> 172.21.15.155:0/2582230771 conn(0x7f82180fc010 msgr2=0x7f82180fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:53.458 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:53.455+0000 7f821d121640 1 -- 172.21.15.155:0/2582230771 shutdown_connections 2024-09-19T06:14:53.458 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:53.455+0000 7f821d121640 1 -- 172.21.15.155:0/2582230771 wait complete. 2024-09-19T06:14:54.112 DEBUG:teuthology.orchestra.run.smithi155:osd.5> sudo journalctl -f -n 0 -u ceph-9f2876d0-764d-11ef-bceb-c7b262605968@osd.5.service 2024-09-19T06:14:54.115 INFO:tasks.cephadm:Deploying osd.6 on smithi155 with /dev/vg_nvme/lv_2... 2024-09-19T06:14:54.115 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T06:14:54.412 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi155/config 2024-09-19T06:14:54.701 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:54 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:54.701 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:54 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:54.701 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:54 smithi155 ceph-mon[35292]: from='osd.5 [v2:172.21.15.155:6808/464507356,v1:172.21.15.155:6809/464507356]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T06:14:54.701 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:54 smithi155 ceph-mon[35292]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T06:14:54.702 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:54 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:54.702 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:54 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:54.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:54 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:54.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:54 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:54.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:54 smithi053 ceph-mon[27968]: from='osd.5 [v2:172.21.15.155:6808/464507356,v1:172.21.15.155:6809/464507356]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T06:14:54.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:54 smithi053 ceph-mon[27968]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T06:14:54.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:54 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:54.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:54 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:14:55.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:55 smithi155 ceph-mon[35292]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T06:14:55.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:55 smithi155 ceph-mon[35292]: osdmap e37: 6 total, 5 up, 6 in 2024-09-19T06:14:55.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:55 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:14:55.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:55 smithi155 ceph-mon[35292]: from='osd.5 [v2:172.21.15.155:6808/464507356,v1:172.21.15.155:6809/464507356]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-19T06:14:55.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:55 smithi155 ceph-mon[35292]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-19T06:14:55.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:55 smithi155 ceph-mon[35292]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 152 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:14:55.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:55 smithi053 ceph-mon[27968]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T06:14:55.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:55 smithi053 ceph-mon[27968]: osdmap e37: 6 total, 5 up, 6 in 2024-09-19T06:14:55.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:55 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:14:55.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:55 smithi053 ceph-mon[27968]: from='osd.5 [v2:172.21.15.155:6808/464507356,v1:172.21.15.155:6809/464507356]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-19T06:14:55.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:55 smithi053 ceph-mon[27968]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-19T06:14:55.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:55 smithi053 ceph-mon[27968]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 152 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:14:56.078 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:14:55 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:14:55.894+0000 7f7e315ce640 -1 osd.5 0 waiting for initial osdmap 2024-09-19T06:14:56.078 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:14:55 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:14:55.898+0000 7f7e293ee640 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:14:56.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:56 smithi155 ceph-mon[35292]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-09-19T06:14:56.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:56 smithi155 ceph-mon[35292]: osdmap e38: 6 total, 5 up, 6 in 2024-09-19T06:14:56.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:56 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:14:56.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:56 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:14:56.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:56 smithi155 ceph-mon[35292]: from='osd.5 ' entity='osd.5' 2024-09-19T06:14:56.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:56 smithi053 ceph-mon[27968]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-09-19T06:14:56.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:56 smithi053 ceph-mon[27968]: osdmap e38: 6 total, 5 up, 6 in 2024-09-19T06:14:56.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:56 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:14:56.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:56 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:14:56.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:56 smithi053 ceph-mon[27968]: from='osd.5 ' entity='osd.5' 2024-09-19T06:14:57.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:57 smithi155 ceph-mon[35292]: purged_snaps scrub starts 2024-09-19T06:14:57.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:57 smithi155 ceph-mon[35292]: purged_snaps scrub ok 2024-09-19T06:14:57.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:57 smithi155 ceph-mon[35292]: osd.5 [v2:172.21.15.155:6808/464507356,v1:172.21.15.155:6809/464507356] boot 2024-09-19T06:14:57.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:57 smithi155 ceph-mon[35292]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T06:14:57.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:57 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:14:57.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:57 smithi155 ceph-mon[35292]: pgmap v131: 1 pgs: 1 remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:14:57.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:57 smithi155 ceph-mon[35292]: osdmap e40: 6 total, 6 up, 6 in 2024-09-19T06:14:57.775 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:14:57.803 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi155:vg_nvme/lv_2 2024-09-19T06:14:57.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:57 smithi053 ceph-mon[27968]: purged_snaps scrub starts 2024-09-19T06:14:57.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:57 smithi053 ceph-mon[27968]: purged_snaps scrub ok 2024-09-19T06:14:57.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:57 smithi053 ceph-mon[27968]: osd.5 [v2:172.21.15.155:6808/464507356,v1:172.21.15.155:6809/464507356] boot 2024-09-19T06:14:57.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:57 smithi053 ceph-mon[27968]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T06:14:57.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:57 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:14:57.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:57 smithi053 ceph-mon[27968]: pgmap v131: 1 pgs: 1 remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:14:57.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:57 smithi053 ceph-mon[27968]: osdmap e40: 6 total, 6 up, 6 in 2024-09-19T06:14:58.315 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi155/config 2024-09-19T06:14:58.844 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.829+0000 7f6d69edf640 1 -- 172.21.15.155:0/1972209996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d64104dc0 msgr2=0x7f6d641071b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:58.844 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.829+0000 7f6d69edf640 1 --2- 172.21.15.155:0/1972209996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d64104dc0 0x7f6d641071b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009880 tx=0x7f6d5802a4a0 comp rx=0 tx=0).stop 2024-09-19T06:14:58.844 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.829+0000 7f6d69edf640 1 -- 172.21.15.155:0/1972209996 shutdown_connections 2024-09-19T06:14:58.844 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.829+0000 7f6d69edf640 1 --2- 172.21.15.155:0/1972209996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d64104dc0 0x7f6d641071b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:58.844 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.829+0000 7f6d69edf640 1 --2- 172.21.15.155:0/1972209996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d64102490 0x7f6d64104880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:58.845 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.829+0000 7f6d69edf640 1 -- 172.21.15.155:0/1972209996 >> 172.21.15.155:0/1972209996 conn(0x7f6d640fc010 msgr2=0x7f6d640fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:58.845 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.829+0000 7f6d69edf640 1 -- 172.21.15.155:0/1972209996 shutdown_connections 2024-09-19T06:14:58.845 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.829+0000 7f6d69edf640 1 -- 172.21.15.155:0/1972209996 wait complete. 2024-09-19T06:14:58.845 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.830+0000 7f6d69edf640 1 Processor -- start 2024-09-19T06:14:58.845 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.830+0000 7f6d69edf640 1 -- start start 2024-09-19T06:14:58.845 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.830+0000 7f6d69edf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d64102490 0x7f6d64197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:58.845 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.830+0000 7f6d69edf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d64104dc0 0x7f6d64197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:58.845 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.830+0000 7f6d69edf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6410a6f0 con 0x7f6d64102490 2024-09-19T06:14:58.845 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.830+0000 7f6d69edf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6410a570 con 0x7f6d64104dc0 2024-09-19T06:14:58.845 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.830+0000 7f6d62ffd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d64104dc0 0x7f6d64197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:58.846 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.830+0000 7f6d62ffd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d64104dc0 0x7f6d64197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:43340/0 (socket says 172.21.15.155:43340) 2024-09-19T06:14:58.846 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.830+0000 7f6d62ffd640 1 -- 172.21.15.155:0/2522451350 learned_addr learned my addr 172.21.15.155:0/2522451350 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-19T06:14:58.846 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.830+0000 7f6d637fe640 1 --2- 172.21.15.155:0/2522451350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d64102490 0x7f6d64197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:58.846 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.830+0000 7f6d62ffd640 1 -- 172.21.15.155:0/2522451350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d64102490 msgr2=0x7f6d64197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:58.846 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.830+0000 7f6d62ffd640 1 --2- 172.21.15.155:0/2522451350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d64102490 0x7f6d64197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:58.846 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.830+0000 7f6d62ffd640 1 -- 172.21.15.155:0/2522451350 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d64197f70 con 0x7f6d64104dc0 2024-09-19T06:14:58.846 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.831+0000 7f6d62ffd640 1 --2- 172.21.15.155:0/2522451350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d64104dc0 0x7f6d64197980 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6d58004650 tx=0x7f6d58004740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:58.846 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.831+0000 7f6d60ff9640 1 -- 172.21.15.155:0/2522451350 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6d5803f070 con 0x7f6d64104dc0 2024-09-19T06:14:58.846 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.831+0000 7f6d69edf640 1 -- 172.21.15.155:0/2522451350 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d6419c980 con 0x7f6d64104dc0 2024-09-19T06:14:58.846 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.831+0000 7f6d60ff9640 1 -- 172.21.15.155:0/2522451350 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d58003aa0 con 0x7f6d64104dc0 2024-09-19T06:14:58.846 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.831+0000 7f6d60ff9640 1 -- 172.21.15.155:0/2522451350 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6d58035030 con 0x7f6d64104dc0 2024-09-19T06:14:58.846 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.831+0000 7f6d69edf640 1 -- 172.21.15.155:0/2522451350 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d6419cf00 con 0x7f6d64104dc0 2024-09-19T06:14:58.847 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.832+0000 7f6d69edf640 1 -- 172.21.15.155:0/2522451350 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d64102870 con 0x7f6d64104dc0 2024-09-19T06:14:58.847 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.833+0000 7f6d60ff9640 1 -- 172.21.15.155:0/2522451350 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f6d58042070 con 0x7f6d64104dc0 2024-09-19T06:14:58.847 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.833+0000 7f6d60ff9640 1 --2- 172.21.15.155:0/2522451350 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f6d4c06e2b0 0x7f6d4c070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:58.847 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.833+0000 7f6d60ff9640 1 -- 172.21.15.155:0/2522451350 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4638+0+0 (secure 0 0 0) 0x7f6d580aeaa0 con 0x7f6d64104dc0 2024-09-19T06:14:58.847 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.833+0000 7f6d637fe640 1 --2- 172.21.15.155:0/2522451350 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f6d4c06e2b0 0x7f6d4c070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:58.847 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.834+0000 7f6d637fe640 1 --2- 172.21.15.155:0/2522451350 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f6d4c06e2b0 0x7f6d4c070770 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6d540058c0 tx=0x7f6d5400a2d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:58.847 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:58.838+0000 7f6d60ff9640 1 -- 172.21.15.155:0/2522451350 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6d58080360 con 0x7f6d64104dc0 2024-09-19T06:14:58.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:58 smithi155 ceph-mon[35292]: osdmap e41: 6 total, 6 up, 6 in 2024-09-19T06:14:58.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:58 smithi155 ceph-mon[35292]: pgmap v134: 1 pgs: 1 remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:14:58.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:58 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:14:59.021 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:14:59.019+0000 7f6d69edf640 1 -- 172.21.15.155:0/2522451350 --> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f6d64105900 con 0x7f6d4c06e2b0 2024-09-19T06:14:59.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:58 smithi053 ceph-mon[27968]: osdmap e41: 6 total, 6 up, 6 in 2024-09-19T06:14:59.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:58 smithi053 ceph-mon[27968]: pgmap v134: 1 pgs: 1 remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:14:59.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:58 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:15:00.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:59 smithi053 ceph-mon[27968]: osdmap e42: 6 total, 6 up, 6 in 2024-09-19T06:15:00.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:59 smithi053 ceph-mon[27968]: from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:15:00.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:15:00.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:15:00.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:15:00.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:59 smithi053 ceph-mon[27968]: Detected new or changed devices on smithi155 2024-09-19T06:15:00.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:00.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:00.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:15:00.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:15:00.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:59 smithi053 ceph-mon[27968]: Adjusting osd_memory_target on smithi155 to 7456M 2024-09-19T06:15:00.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:00.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:15:00.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:15:00.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:00.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:15:00.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:15:00.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:15:00.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:14:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:00.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:59 smithi155 ceph-mon[35292]: osdmap e42: 6 total, 6 up, 6 in 2024-09-19T06:15:00.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:59 smithi155 ceph-mon[35292]: from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:15:00.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:15:00.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:15:00.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:15:00.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:59 smithi155 ceph-mon[35292]: Detected new or changed devices on smithi155 2024-09-19T06:15:00.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:00.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:00.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:15:00.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:15:00.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:59 smithi155 ceph-mon[35292]: Adjusting osd_memory_target on smithi155 to 7456M 2024-09-19T06:15:00.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:00.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:15:00.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:15:00.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:00.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:15:00.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:15:00.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:15:00.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:14:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:00.826 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:00 smithi155 ceph-mon[35292]: osdmap e43: 6 total, 6 up, 6 in 2024-09-19T06:15:00.826 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:00 smithi155 ceph-mon[35292]: pgmap v137: 1 pgs: 1 remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:15:01.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:00 smithi053 ceph-mon[27968]: osdmap e43: 6 total, 6 up, 6 in 2024-09-19T06:15:01.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:00 smithi053 ceph-mon[27968]: pgmap v137: 1 pgs: 1 remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:15:01.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:01 smithi155 ceph-mon[35292]: from='client.? 172.21.15.155:0/1185474186' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc8a88f5-75ab-4482-88e0-5cc5e6f06db4"}]: dispatch 2024-09-19T06:15:01.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:01 smithi155 ceph-mon[35292]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc8a88f5-75ab-4482-88e0-5cc5e6f06db4"}]: dispatch 2024-09-19T06:15:01.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:01 smithi155 ceph-mon[35292]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bc8a88f5-75ab-4482-88e0-5cc5e6f06db4"}]': finished 2024-09-19T06:15:01.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:01 smithi155 ceph-mon[35292]: osdmap e44: 7 total, 6 up, 7 in 2024-09-19T06:15:01.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:01 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:15:02.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:01 smithi053 ceph-mon[27968]: from='client.? 172.21.15.155:0/1185474186' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc8a88f5-75ab-4482-88e0-5cc5e6f06db4"}]: dispatch 2024-09-19T06:15:02.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:01 smithi053 ceph-mon[27968]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc8a88f5-75ab-4482-88e0-5cc5e6f06db4"}]: dispatch 2024-09-19T06:15:02.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:01 smithi053 ceph-mon[27968]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bc8a88f5-75ab-4482-88e0-5cc5e6f06db4"}]': finished 2024-09-19T06:15:02.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:01 smithi053 ceph-mon[27968]: osdmap e44: 7 total, 6 up, 7 in 2024-09-19T06:15:02.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:01 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:15:03.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:02 smithi053 ceph-mon[27968]: from='client.? 172.21.15.155:0/1930160096' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:15:03.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:02 smithi053 ceph-mon[27968]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-09-19T06:15:03.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:02 smithi155 ceph-mon[35292]: from='client.? 172.21.15.155:0/1930160096' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:15:03.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:02 smithi155 ceph-mon[35292]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-09-19T06:15:05.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:04 smithi053 ceph-mon[27968]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T06:15:05.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:04 smithi155 ceph-mon[35292]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T06:15:06.964 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:06 smithi155 ceph-mon[35292]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T06:15:07.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:06 smithi053 ceph-mon[27968]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T06:15:08.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:08 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T06:15:08.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:08 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:15:08.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:08 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T06:15:08.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:08 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:15:09.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:09 smithi053 ceph-mon[27968]: Deploying daemon osd.6 on smithi155 2024-09-19T06:15:09.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:09 smithi053 ceph-mon[27968]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-19T06:15:09.868 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:09 smithi155 ceph-mon[35292]: Deploying daemon osd.6 on smithi155 2024-09-19T06:15:09.868 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:09 smithi155 ceph-mon[35292]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-19T06:15:11.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:10 smithi053 ceph-mon[27968]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T06:15:11.190 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:10 smithi155 ceph-mon[35292]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T06:15:13.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:12 smithi053 ceph-mon[27968]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:15:13.198 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:12 smithi155 ceph-mon[35292]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:15:14.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:13 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:15:14.112 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:13 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:15:15.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:14 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:15:15.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:14 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:15.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:14 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:15.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:14 smithi053 ceph-mon[27968]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:15:15.155 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:14 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:15:15.155 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:14 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:15.155 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:14 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:15.155 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:14 smithi155 ceph-mon[35292]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:15:16.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:16 smithi155 ceph-mon[35292]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:15:17.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:16 smithi053 ceph-mon[27968]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:15:17.794 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 6 on host 'smithi155' 2024-09-19T06:15:17.795 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:17.792+0000 7f6d60ff9640 1 -- 172.21.15.155:0/2522451350 <== mgr.14217 v2:172.21.15.53:6800/3884355752 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6d64105900 con 0x7f6d4c06e2b0 2024-09-19T06:15:17.797 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:17.795+0000 7f6d69edf640 1 -- 172.21.15.155:0/2522451350 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f6d4c06e2b0 msgr2=0x7f6d4c070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:17.797 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:17.795+0000 7f6d69edf640 1 --2- 172.21.15.155:0/2522451350 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f6d4c06e2b0 0x7f6d4c070770 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6d540058c0 tx=0x7f6d5400a2d0 comp rx=0 tx=0).stop 2024-09-19T06:15:17.797 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:17.795+0000 7f6d69edf640 1 -- 172.21.15.155:0/2522451350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d64104dc0 msgr2=0x7f6d64197980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:17.797 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:17.795+0000 7f6d69edf640 1 --2- 172.21.15.155:0/2522451350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d64104dc0 0x7f6d64197980 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6d58004650 tx=0x7f6d58004740 comp rx=0 tx=0).stop 2024-09-19T06:15:17.797 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:17.795+0000 7f6d69edf640 1 -- 172.21.15.155:0/2522451350 shutdown_connections 2024-09-19T06:15:17.797 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:17.795+0000 7f6d69edf640 1 --2- 172.21.15.155:0/2522451350 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f6d4c06e2b0 0x7f6d4c070770 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:17.797 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:17.795+0000 7f6d69edf640 1 --2- 172.21.15.155:0/2522451350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d64104dc0 0x7f6d64197980 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:17.797 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:17.795+0000 7f6d69edf640 1 --2- 172.21.15.155:0/2522451350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d64102490 0x7f6d64197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:17.797 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:17.795+0000 7f6d69edf640 1 -- 172.21.15.155:0/2522451350 >> 172.21.15.155:0/2522451350 conn(0x7f6d640fc010 msgr2=0x7f6d640fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:17.797 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:17.795+0000 7f6d69edf640 1 -- 172.21.15.155:0/2522451350 shutdown_connections 2024-09-19T06:15:17.797 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:17.795+0000 7f6d69edf640 1 -- 172.21.15.155:0/2522451350 wait complete. 2024-09-19T06:15:18.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:18 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:18.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:18 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:18.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:18 smithi155 ceph-mon[35292]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:15:19.020 DEBUG:teuthology.orchestra.run.smithi155:osd.6> sudo journalctl -f -n 0 -u ceph-9f2876d0-764d-11ef-bceb-c7b262605968@osd.6.service 2024-09-19T06:15:19.023 INFO:tasks.cephadm:Deploying osd.7 on smithi155 with /dev/vg_nvme/lv_1... 2024-09-19T06:15:19.023 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T06:15:19.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:18 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:19.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:18 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:19.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:18 smithi053 ceph-mon[27968]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:15:19.327 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi155/config 2024-09-19T06:15:19.983 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:15:19 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:15:19.638+0000 7fef51a91640 -1 osd.6 0 log_to_monitors true 2024-09-19T06:15:20.594 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:20 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:20.594 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:20 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:20.594 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:20 smithi155 ceph-mon[35292]: from='osd.6 [v2:172.21.15.155:6816/2574655677,v1:172.21.15.155:6817/2574655677]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T06:15:20.594 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:20 smithi155 ceph-mon[35292]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T06:15:20.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:20 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:20.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:20 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:20.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:20 smithi053 ceph-mon[27968]: from='osd.6 [v2:172.21.15.155:6816/2574655677,v1:172.21.15.155:6817/2574655677]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T06:15:20.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:20 smithi053 ceph-mon[27968]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T06:15:21.482 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:15:21 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:15:21.331+0000 7fef4da14640 -1 osd.6 0 waiting for initial osdmap 2024-09-19T06:15:21.483 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:15:21 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:15:21.338+0000 7fef4983c640 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:15:21.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:21 smithi155 ceph-mon[35292]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T06:15:21.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:21 smithi155 ceph-mon[35292]: osdmap e45: 7 total, 6 up, 7 in 2024-09-19T06:15:21.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:21 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:15:21.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:21 smithi155 ceph-mon[35292]: from='osd.6 [v2:172.21.15.155:6816/2574655677,v1:172.21.15.155:6817/2574655677]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-19T06:15:21.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:21 smithi155 ceph-mon[35292]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-19T06:15:21.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:21 smithi155 ceph-mon[35292]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:15:21.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:21 smithi053 ceph-mon[27968]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T06:15:21.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:21 smithi053 ceph-mon[27968]: osdmap e45: 7 total, 6 up, 7 in 2024-09-19T06:15:21.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:21 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:15:21.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:21 smithi053 ceph-mon[27968]: from='osd.6 [v2:172.21.15.155:6816/2574655677,v1:172.21.15.155:6817/2574655677]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-19T06:15:21.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:21 smithi053 ceph-mon[27968]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-19T06:15:21.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:21 smithi053 ceph-mon[27968]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:15:22.147 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:15:22.174 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi155:vg_nvme/lv_1 2024-09-19T06:15:22.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:22 smithi155 ceph-mon[35292]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-09-19T06:15:22.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:22 smithi155 ceph-mon[35292]: osdmap e46: 7 total, 6 up, 7 in 2024-09-19T06:15:22.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:22 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:15:22.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:22 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:15:22.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:22 smithi155 ceph-mon[35292]: osd.6 [v2:172.21.15.155:6816/2574655677,v1:172.21.15.155:6817/2574655677] boot 2024-09-19T06:15:22.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:22 smithi155 ceph-mon[35292]: osdmap e47: 7 total, 7 up, 7 in 2024-09-19T06:15:22.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:22 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:15:22.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:22 smithi053 ceph-mon[27968]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-09-19T06:15:22.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:22 smithi053 ceph-mon[27968]: osdmap e46: 7 total, 6 up, 7 in 2024-09-19T06:15:22.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:22 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:15:22.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:22 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:15:22.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:22 smithi053 ceph-mon[27968]: osd.6 [v2:172.21.15.155:6816/2574655677,v1:172.21.15.155:6817/2574655677] boot 2024-09-19T06:15:22.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:22 smithi053 ceph-mon[27968]: osdmap e47: 7 total, 7 up, 7 in 2024-09-19T06:15:22.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:22 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:15:23.160 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi155/config 2024-09-19T06:15:23.561 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:23 smithi155 ceph-mon[35292]: purged_snaps scrub starts 2024-09-19T06:15:23.561 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:23 smithi155 ceph-mon[35292]: purged_snaps scrub ok 2024-09-19T06:15:23.561 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:23 smithi155 ceph-mon[35292]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:15:23.562 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:23 smithi155 ceph-mon[35292]: osdmap e48: 7 total, 7 up, 7 in 2024-09-19T06:15:23.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:23 smithi053 ceph-mon[27968]: purged_snaps scrub starts 2024-09-19T06:15:23.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:23 smithi053 ceph-mon[27968]: purged_snaps scrub ok 2024-09-19T06:15:23.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:23 smithi053 ceph-mon[27968]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:15:23.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:23 smithi053 ceph-mon[27968]: osdmap e48: 7 total, 7 up, 7 in 2024-09-19T06:15:23.669 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.643+0000 7fdf61558640 1 -- 172.21.15.155:0/2636899221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf5c102c20 msgr2=0x7fdf5c1030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:23.669 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.643+0000 7fdf61558640 1 --2- 172.21.15.155:0/2636899221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf5c102c20 0x7fdf5c1030a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c009880 tx=0x7fdf4c02a4a0 comp rx=0 tx=0).stop 2024-09-19T06:15:23.670 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.643+0000 7fdf61558640 1 -- 172.21.15.155:0/2636899221 shutdown_connections 2024-09-19T06:15:23.670 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.643+0000 7fdf61558640 1 --2- 172.21.15.155:0/2636899221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf5c102c20 0x7fdf5c1030a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:23.670 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.643+0000 7fdf61558640 1 --2- 172.21.15.155:0/2636899221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf5c0ffc40 0x7fdf5c100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:23.670 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.643+0000 7fdf61558640 1 -- 172.21.15.155:0/2636899221 >> 172.21.15.155:0/2636899221 conn(0x7fdf5c0fb790 msgr2=0x7fdf5c0fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:23.670 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.643+0000 7fdf61558640 1 -- 172.21.15.155:0/2636899221 shutdown_connections 2024-09-19T06:15:23.670 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.643+0000 7fdf61558640 1 -- 172.21.15.155:0/2636899221 wait complete. 2024-09-19T06:15:23.670 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.644+0000 7fdf61558640 1 Processor -- start 2024-09-19T06:15:23.670 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.644+0000 7fdf61558640 1 -- start start 2024-09-19T06:15:23.670 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.644+0000 7fdf61558640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf5c0ffc40 0x7fdf5c197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:23.670 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.644+0000 7fdf61558640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf5c102c20 0x7fdf5c197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:23.670 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.644+0000 7fdf61558640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c10a710 con 0x7fdf5c0ffc40 2024-09-19T06:15:23.670 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.644+0000 7fdf61558640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c10a590 con 0x7fdf5c102c20 2024-09-19T06:15:23.671 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.644+0000 7fdf5a7fc640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf5c102c20 0x7fdf5c197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:23.671 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.644+0000 7fdf5a7fc640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf5c102c20 0x7fdf5c197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:57032/0 (socket says 172.21.15.155:57032) 2024-09-19T06:15:23.671 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.644+0000 7fdf5a7fc640 1 -- 172.21.15.155:0/2031315942 learned_addr learned my addr 172.21.15.155:0/2031315942 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-19T06:15:23.671 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.645+0000 7fdf5a7fc640 1 -- 172.21.15.155:0/2031315942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf5c0ffc40 msgr2=0x7fdf5c197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:23.671 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.645+0000 7fdf5a7fc640 1 --2- 172.21.15.155:0/2031315942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf5c0ffc40 0x7fdf5c197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:23.671 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.645+0000 7fdf5a7fc640 1 -- 172.21.15.155:0/2031315942 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5c197f80 con 0x7fdf5c102c20 2024-09-19T06:15:23.671 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.645+0000 7fdf5a7fc640 1 --2- 172.21.15.155:0/2031315942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf5c102c20 0x7fdf5c197990 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c004e80 tx=0x7fdf4c0045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:23.671 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.645+0000 7fdf43fff640 1 -- 172.21.15.155:0/2031315942 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdf4c03f070 con 0x7fdf5c102c20 2024-09-19T06:15:23.671 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.645+0000 7fdf61558640 1 -- 172.21.15.155:0/2031315942 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf5c19c990 con 0x7fdf5c102c20 2024-09-19T06:15:23.671 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.645+0000 7fdf43fff640 1 -- 172.21.15.155:0/2031315942 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf4c0049d0 con 0x7fdf5c102c20 2024-09-19T06:15:23.671 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.645+0000 7fdf43fff640 1 -- 172.21.15.155:0/2031315942 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdf4c035030 con 0x7fdf5c102c20 2024-09-19T06:15:23.671 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.645+0000 7fdf61558640 1 -- 172.21.15.155:0/2031315942 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf5c19cf10 con 0x7fdf5c102c20 2024-09-19T06:15:23.671 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.646+0000 7fdf61558640 1 -- 172.21.15.155:0/2031315942 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf5c100040 con 0x7fdf5c102c20 2024-09-19T06:15:23.672 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.647+0000 7fdf43fff640 1 -- 172.21.15.155:0/2031315942 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7fdf4c042070 con 0x7fdf5c102c20 2024-09-19T06:15:23.672 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.647+0000 7fdf43fff640 1 --2- 172.21.15.155:0/2031315942 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fdf4406e2b0 0x7fdf44070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:23.672 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.647+0000 7fdf43fff640 1 -- 172.21.15.155:0/2031315942 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5099+0+0 (secure 0 0 0) 0x7fdf4c0aebb0 con 0x7fdf5c102c20 2024-09-19T06:15:23.672 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.648+0000 7fdf5affd640 1 --2- 172.21.15.155:0/2031315942 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fdf4406e2b0 0x7fdf44070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:23.672 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.648+0000 7fdf5affd640 1 --2- 172.21.15.155:0/2031315942 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fdf4406e2b0 0x7fdf44070770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c069fb0 tx=0x7fdf50009630 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:23.672 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.652+0000 7fdf43fff640 1 -- 172.21.15.155:0/2031315942 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdf4c0b5080 con 0x7fdf5c102c20 2024-09-19T06:15:23.838 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:23.835+0000 7fdf61558640 1 -- 172.21.15.155:0/2031315942 --> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fdf5c02c640 con 0x7fdf4406e2b0 2024-09-19T06:15:25.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:24 smithi053 ceph-mon[27968]: osdmap e49: 7 total, 7 up, 7 in 2024-09-19T06:15:25.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:24 smithi053 ceph-mon[27968]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:15:25.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:24 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:15:25.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:24 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:15:25.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:24 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:15:25.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:24 smithi053 ceph-mon[27968]: Detected new or changed devices on smithi155 2024-09-19T06:15:25.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:24 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:25.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:24 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:25.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:24 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:15:25.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:24 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:15:25.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:24 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:15:25.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:24 smithi053 ceph-mon[27968]: Adjusting osd_memory_target on smithi155 to 4970M 2024-09-19T06:15:25.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:24 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:25.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:24 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:15:25.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:24 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:15:25.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:24 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:25.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:24 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:15:25.107 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:24 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:15:25.107 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:24 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:15:25.107 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:24 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:25.107 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:24 smithi053 ceph-mon[27968]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:15:25.146 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:24 smithi155 ceph-mon[35292]: osdmap e49: 7 total, 7 up, 7 in 2024-09-19T06:15:25.146 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:24 smithi155 ceph-mon[35292]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:15:25.146 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:24 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:15:25.146 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:24 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:15:25.146 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:24 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:15:25.146 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:24 smithi155 ceph-mon[35292]: Detected new or changed devices on smithi155 2024-09-19T06:15:25.147 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:24 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:25.147 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:24 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:25.147 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:24 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:15:25.147 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:24 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:15:25.147 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:24 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:15:25.147 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:24 smithi155 ceph-mon[35292]: Adjusting osd_memory_target on smithi155 to 4970M 2024-09-19T06:15:25.147 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:24 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:25.147 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:24 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:15:25.147 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:24 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:15:25.147 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:24 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:25.147 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:24 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:15:25.147 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:24 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:15:25.147 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:24 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:15:25.147 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:24 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:25.148 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:24 smithi155 ceph-mon[35292]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:15:26.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:26 smithi155 ceph-mon[35292]: osdmap e50: 7 total, 7 up, 7 in 2024-09-19T06:15:26.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:26 smithi053 ceph-mon[27968]: osdmap e50: 7 total, 7 up, 7 in 2024-09-19T06:15:27.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:27 smithi053 ceph-mon[27968]: osdmap e51: 7 total, 7 up, 7 in 2024-09-19T06:15:27.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:27 smithi053 ceph-mon[27968]: from='client.? 172.21.15.155:0/2090476887' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b91759fe-f27c-4ab2-9bd3-b5961b358d2a"}]: dispatch 2024-09-19T06:15:27.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:27 smithi053 ceph-mon[27968]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b91759fe-f27c-4ab2-9bd3-b5961b358d2a"}]: dispatch 2024-09-19T06:15:27.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:27 smithi053 ceph-mon[27968]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b91759fe-f27c-4ab2-9bd3-b5961b358d2a"}]': finished 2024-09-19T06:15:27.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:27 smithi053 ceph-mon[27968]: osdmap e52: 8 total, 7 up, 8 in 2024-09-19T06:15:27.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:27 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:15:27.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:27 smithi053 ceph-mon[27968]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:15:27.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:27 smithi053 ceph-mon[27968]: from='client.? 172.21.15.155:0/3677018254' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:15:27.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:27 smithi155 ceph-mon[35292]: osdmap e51: 7 total, 7 up, 7 in 2024-09-19T06:15:27.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:27 smithi155 ceph-mon[35292]: from='client.? 172.21.15.155:0/2090476887' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b91759fe-f27c-4ab2-9bd3-b5961b358d2a"}]: dispatch 2024-09-19T06:15:27.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:27 smithi155 ceph-mon[35292]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b91759fe-f27c-4ab2-9bd3-b5961b358d2a"}]: dispatch 2024-09-19T06:15:27.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:27 smithi155 ceph-mon[35292]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b91759fe-f27c-4ab2-9bd3-b5961b358d2a"}]': finished 2024-09-19T06:15:27.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:27 smithi155 ceph-mon[35292]: osdmap e52: 8 total, 7 up, 8 in 2024-09-19T06:15:27.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:27 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:15:27.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:27 smithi155 ceph-mon[35292]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:15:27.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:27 smithi155 ceph-mon[35292]: from='client.? 172.21.15.155:0/3677018254' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:15:28.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:28 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:15:28.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:28 smithi155 ceph-mon[35292]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 207 MiB used, 626 GiB / 626 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-09-19T06:15:29.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:28 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:15:29.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:28 smithi053 ceph-mon[27968]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 207 MiB used, 626 GiB / 626 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-09-19T06:15:31.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:30 smithi053 ceph-mon[27968]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 207 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T06:15:31.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:30 smithi155 ceph-mon[35292]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 207 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T06:15:33.015 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:32 smithi155 ceph-mon[35292]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-19T06:15:33.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:32 smithi053 ceph-mon[27968]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-19T06:15:34.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:34 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T06:15:34.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:34 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:15:34.381 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:34 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T06:15:34.381 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:34 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:15:35.315 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:35 smithi155 ceph-mon[35292]: Deploying daemon osd.7 on smithi155 2024-09-19T06:15:35.315 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:35 smithi155 ceph-mon[35292]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-19T06:15:35.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:35 smithi053 ceph-mon[27968]: Deploying daemon osd.7 on smithi155 2024-09-19T06:15:35.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:35 smithi053 ceph-mon[27968]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-19T06:15:37.083 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:36 smithi155 ceph-mon[35292]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-19T06:15:37.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:36 smithi053 ceph-mon[27968]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-19T06:15:38.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:38 smithi155 ceph-mon[35292]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T06:15:39.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:38 smithi053 ceph-mon[27968]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T06:15:39.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:39 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:15:39.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:39 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:39.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:39 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:40.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:39 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:15:40.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:39 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:40.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:39 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:40.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:40 smithi155 ceph-mon[35292]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:15:41.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:40 smithi053 ceph-mon[27968]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:15:43.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:42 smithi053 ceph-mon[27968]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:15:43.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:42 smithi155 ceph-mon[35292]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:15:43.559 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:43.557+0000 7fdf43fff640 1 -- 172.21.15.155:0/2031315942 <== mgr.14217 v2:172.21.15.53:6800/3884355752 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdf5c02c640 con 0x7fdf4406e2b0 2024-09-19T06:15:43.559 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 7 on host 'smithi155' 2024-09-19T06:15:43.561 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:43.559+0000 7fdf61558640 1 -- 172.21.15.155:0/2031315942 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fdf4406e2b0 msgr2=0x7fdf44070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:43.561 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:43.559+0000 7fdf61558640 1 --2- 172.21.15.155:0/2031315942 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fdf4406e2b0 0x7fdf44070770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c069fb0 tx=0x7fdf50009630 comp rx=0 tx=0).stop 2024-09-19T06:15:43.561 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:43.559+0000 7fdf61558640 1 -- 172.21.15.155:0/2031315942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf5c102c20 msgr2=0x7fdf5c197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:43.561 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:43.559+0000 7fdf61558640 1 --2- 172.21.15.155:0/2031315942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf5c102c20 0x7fdf5c197990 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c004e80 tx=0x7fdf4c0045e0 comp rx=0 tx=0).stop 2024-09-19T06:15:43.561 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:43.559+0000 7fdf61558640 1 -- 172.21.15.155:0/2031315942 shutdown_connections 2024-09-19T06:15:43.561 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:43.559+0000 7fdf61558640 1 --2- 172.21.15.155:0/2031315942 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fdf4406e2b0 0x7fdf44070770 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:43.561 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:43.559+0000 7fdf61558640 1 --2- 172.21.15.155:0/2031315942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf5c102c20 0x7fdf5c197990 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:43.561 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:43.559+0000 7fdf61558640 1 --2- 172.21.15.155:0/2031315942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf5c0ffc40 0x7fdf5c197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:43.561 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:43.559+0000 7fdf61558640 1 -- 172.21.15.155:0/2031315942 >> 172.21.15.155:0/2031315942 conn(0x7fdf5c0fb790 msgr2=0x7fdf5c1061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:43.561 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:43.559+0000 7fdf61558640 1 -- 172.21.15.155:0/2031315942 shutdown_connections 2024-09-19T06:15:43.561 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:43.559+0000 7fdf61558640 1 -- 172.21.15.155:0/2031315942 wait complete. 2024-09-19T06:15:44.214 DEBUG:teuthology.orchestra.run.smithi155:osd.7> sudo journalctl -f -n 0 -u ceph-9f2876d0-764d-11ef-bceb-c7b262605968@osd.7.service 2024-09-19T06:15:44.216 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-19T06:15:44.216 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T06:15:44.498 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:15:44.803 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:44 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:44.803 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:44 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:44.803 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:44 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:15:44.803 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:44 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:44.803 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:44 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:44.833 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:44 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:44.833 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:44 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:44.833 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:44 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:15:44.833 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:44 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:44.833 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:44 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:44.833 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:15:44 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:15:44.624+0000 7fe1027fd640 -1 osd.7 0 log_to_monitors true 2024-09-19T06:15:44.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.968+0000 7fc9b15fb640 1 -- 172.21.15.53:0/3984762848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9ac100860 msgr2=0x7fc9ac100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:44.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.968+0000 7fc9b15fb640 1 --2- 172.21.15.53:0/3984762848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9ac100860 0x7fc9ac100c60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009c20 tx=0x7fc99c02a970 comp rx=0 tx=0).stop 2024-09-19T06:15:44.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.968+0000 7fc9b15fb640 1 -- 172.21.15.53:0/3984762848 shutdown_connections 2024-09-19T06:15:44.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.968+0000 7fc9b15fb640 1 --2- 172.21.15.53:0/3984762848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9ac101a60 0x7fc9ac101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:44.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.968+0000 7fc9b15fb640 1 --2- 172.21.15.53:0/3984762848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9ac100860 0x7fc9ac100c60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:44.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.968+0000 7fc9b15fb640 1 -- 172.21.15.53:0/3984762848 >> 172.21.15.53:0/3984762848 conn(0x7fc9ac0fc010 msgr2=0x7fc9ac0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:44.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.968+0000 7fc9b15fb640 1 -- 172.21.15.53:0/3984762848 shutdown_connections 2024-09-19T06:15:44.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.968+0000 7fc9b15fb640 1 -- 172.21.15.53:0/3984762848 wait complete. 2024-09-19T06:15:44.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.968+0000 7fc9b15fb640 1 Processor -- start 2024-09-19T06:15:44.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.969+0000 7fc9b15fb640 1 -- start start 2024-09-19T06:15:44.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.969+0000 7fc9b15fb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9ac100860 0x7fc9ac199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:44.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.969+0000 7fc9b15fb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9ac101a60 0x7fc9ac199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:44.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.969+0000 7fc9b15fb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac1095f0 con 0x7fc9ac100860 2024-09-19T06:15:44.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.969+0000 7fc9b15fb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac109470 con 0x7fc9ac101a60 2024-09-19T06:15:44.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.969+0000 7fc9aaffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9ac100860 0x7fc9ac199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:44.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.969+0000 7fc9aaffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9ac100860 0x7fc9ac199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34606/0 (socket says 172.21.15.53:34606) 2024-09-19T06:15:44.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.969+0000 7fc9aaffd640 1 -- 172.21.15.53:0/2839359226 learned_addr learned my addr 172.21.15.53:0/2839359226 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:15:44.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.969+0000 7fc9aa7fc640 1 --2- 172.21.15.53:0/2839359226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9ac101a60 0x7fc9ac199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:44.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.969+0000 7fc9aaffd640 1 -- 172.21.15.53:0/2839359226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9ac101a60 msgr2=0x7fc9ac199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:44.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.969+0000 7fc9aaffd640 1 --2- 172.21.15.53:0/2839359226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9ac101a60 0x7fc9ac199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:44.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.969+0000 7fc9aaffd640 1 -- 172.21.15.53:0/2839359226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9ac19a160 con 0x7fc9ac100860 2024-09-19T06:15:44.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.969+0000 7fc9aaffd640 1 --2- 172.21.15.53:0/2839359226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9ac100860 0x7fc9ac199630 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc99c006010 tx=0x7fc99c02fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:44.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.970+0000 7fc993fff640 1 -- 172.21.15.53:0/2839359226 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc99c047070 con 0x7fc9ac100860 2024-09-19T06:15:44.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.970+0000 7fc9b15fb640 1 -- 172.21.15.53:0/2839359226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9ac19eb70 con 0x7fc9ac100860 2024-09-19T06:15:44.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.970+0000 7fc993fff640 1 -- 172.21.15.53:0/2839359226 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc99c003e20 con 0x7fc9ac100860 2024-09-19T06:15:44.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.970+0000 7fc993fff640 1 -- 172.21.15.53:0/2839359226 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc99c034070 con 0x7fc9ac100860 2024-09-19T06:15:44.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.970+0000 7fc9b15fb640 1 -- 172.21.15.53:0/2839359226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9ac19f070 con 0x7fc9ac100860 2024-09-19T06:15:44.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.971+0000 7fc9b15fb640 1 -- 172.21.15.53:0/2839359226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9ac100c60 con 0x7fc9ac100860 2024-09-19T06:15:44.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.971+0000 7fc993fff640 1 -- 172.21.15.53:0/2839359226 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7fc99c04a070 con 0x7fc9ac100860 2024-09-19T06:15:44.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.972+0000 7fc993fff640 1 --2- 172.21.15.53:0/2839359226 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fc99406e2b0 0x7fc994070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:44.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.972+0000 7fc993fff640 1 -- 172.21.15.53:0/2839359226 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5185+0+0 (secure 0 0 0) 0x7fc99c0aeef0 con 0x7fc9ac100860 2024-09-19T06:15:44.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.972+0000 7fc9aa7fc640 1 --2- 172.21.15.53:0/2839359226 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fc99406e2b0 0x7fc994070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:44.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.972+0000 7fc9aa7fc640 1 --2- 172.21.15.53:0/2839359226 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fc99406e2b0 0x7fc994070770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac19aae0 tx=0x7fc9a000b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:44.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:44.976+0000 7fc993fff640 1 -- 172.21.15.53:0/2839359226 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc99c03f070 con 0x7fc9ac100860 2024-09-19T06:15:45.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:45.154+0000 7fc9b15fb640 1 -- 172.21.15.53:0/2839359226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fc9ac106720 con 0x7fc9ac100860 2024-09-19T06:15:45.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:45.155+0000 7fc993fff640 1 -- 172.21.15.53:0/2839359226 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v52) ==== 74+0+130 (secure 0 0 0) 0x7fc99c080580 con 0x7fc9ac100860 2024-09-19T06:15:45.159 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:15:45.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:45.157+0000 7fc9b15fb640 1 -- 172.21.15.53:0/2839359226 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fc99406e2b0 msgr2=0x7fc994070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:45.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:45.157+0000 7fc9b15fb640 1 --2- 172.21.15.53:0/2839359226 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fc99406e2b0 0x7fc994070770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac19aae0 tx=0x7fc9a000b410 comp rx=0 tx=0).stop 2024-09-19T06:15:45.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:45.157+0000 7fc9b15fb640 1 -- 172.21.15.53:0/2839359226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9ac100860 msgr2=0x7fc9ac199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:45.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:45.157+0000 7fc9b15fb640 1 --2- 172.21.15.53:0/2839359226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9ac100860 0x7fc9ac199630 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc99c006010 tx=0x7fc99c02fc50 comp rx=0 tx=0).stop 2024-09-19T06:15:45.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:45.157+0000 7fc9b15fb640 1 -- 172.21.15.53:0/2839359226 shutdown_connections 2024-09-19T06:15:45.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:45.157+0000 7fc9b15fb640 1 --2- 172.21.15.53:0/2839359226 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fc99406e2b0 0x7fc994070770 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:45.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:45.157+0000 7fc9b15fb640 1 --2- 172.21.15.53:0/2839359226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9ac101a60 0x7fc9ac199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:45.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:45.157+0000 7fc9b15fb640 1 --2- 172.21.15.53:0/2839359226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9ac100860 0x7fc9ac199630 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:45.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:45.157+0000 7fc9b15fb640 1 -- 172.21.15.53:0/2839359226 >> 172.21.15.53:0/2839359226 conn(0x7fc9ac0fc010 msgr2=0x7fc9ac0fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:45.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:45.157+0000 7fc9b15fb640 1 -- 172.21.15.53:0/2839359226 shutdown_connections 2024-09-19T06:15:45.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:45.157+0000 7fc9b15fb640 1 -- 172.21.15.53:0/2839359226 wait complete. 2024-09-19T06:15:45.734 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:45 smithi155 ceph-mon[35292]: from='osd.7 [v2:172.21.15.155:6824/2269161534,v1:172.21.15.155:6825/2269161534]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T06:15:45.734 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:45 smithi155 ceph-mon[35292]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T06:15:45.734 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:45 smithi155 ceph-mon[35292]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:15:45.735 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:45 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/2839359226' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T06:15:45.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:45 smithi053 ceph-mon[27968]: from='osd.7 [v2:172.21.15.155:6824/2269161534,v1:172.21.15.155:6825/2269161534]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T06:15:45.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:45 smithi053 ceph-mon[27968]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T06:15:45.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:45 smithi053 ceph-mon[27968]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:15:45.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:45 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/2839359226' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T06:15:45.923 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":52,"num_osds":8,"num_up_osds":7,"osd_up_since":1726726521,"num_in_osds":8,"osd_in_since":1726726526,"num_remapped_pgs":0} 2024-09-19T06:15:46.827 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:46 smithi155 ceph-mon[35292]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T06:15:46.827 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:46 smithi155 ceph-mon[35292]: osdmap e53: 8 total, 7 up, 8 in 2024-09-19T06:15:46.827 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:46 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:15:46.827 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:46 smithi155 ceph-mon[35292]: from='osd.7 [v2:172.21.15.155:6824/2269161534,v1:172.21.15.155:6825/2269161534]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-19T06:15:46.827 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:46 smithi155 ceph-mon[35292]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-19T06:15:46.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:46 smithi053 ceph-mon[27968]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T06:15:46.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:46 smithi053 ceph-mon[27968]: osdmap e53: 8 total, 7 up, 8 in 2024-09-19T06:15:46.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:46 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:15:46.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:46 smithi053 ceph-mon[27968]: from='osd.7 [v2:172.21.15.155:6824/2269161534,v1:172.21.15.155:6825/2269161534]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-19T06:15:46.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:46 smithi053 ceph-mon[27968]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-19T06:15:46.925 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T06:15:47.201 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:15:47.204 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:15:46 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:15:46.992+0000 7fe0fe780640 -1 osd.7 0 waiting for initial osdmap 2024-09-19T06:15:47.204 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:15:47 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:15:46.999+0000 7fe0f65a0640 -1 osd.7 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:15:47.574 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:47 smithi053 ceph-mon[27968]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-09-19T06:15:47.574 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:47 smithi053 ceph-mon[27968]: osdmap e54: 8 total, 7 up, 8 in 2024-09-19T06:15:47.574 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:47 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:15:47.574 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:47 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:15:47.574 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:47 smithi053 ceph-mon[27968]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:15:47.574 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:47 smithi053 ceph-mon[27968]: osdmap e55: 8 total, 7 up, 8 in 2024-09-19T06:15:47.574 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:47 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:15:47.574 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:47 smithi053 ceph-mon[27968]: from='osd.7 ' entity='osd.7' 2024-09-19T06:15:47.574 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:47 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:47.574 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:47 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:47.574 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:47 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:15:47.574 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:47 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:15:47.575 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:47 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:15:47.575 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:47 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:15:47.575 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:47 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:47.575 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:47 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:15:47.575 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:47 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:15:47.575 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:47 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:47.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.658+0000 7f3c769da640 1 -- 172.21.15.53:0/2577732558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c70109e20 msgr2=0x7f3c70073410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:47.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.658+0000 7f3c769da640 1 --2- 172.21.15.53:0/2577732558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c70109e20 0x7f3c70073410 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009880 tx=0x7f3c6402a4a0 comp rx=0 tx=0).stop 2024-09-19T06:15:47.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.658+0000 7f3c769da640 1 -- 172.21.15.53:0/2577732558 shutdown_connections 2024-09-19T06:15:47.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.658+0000 7f3c769da640 1 --2- 172.21.15.53:0/2577732558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c70109e20 0x7f3c70073410 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:47.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.658+0000 7f3c769da640 1 --2- 172.21.15.53:0/2577732558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c70109450 0x7f3c70109850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:47.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.658+0000 7f3c769da640 1 -- 172.21.15.53:0/2577732558 >> 172.21.15.53:0/2577732558 conn(0x7f3c7006ea50 msgr2=0x7f3c70070e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:47.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.658+0000 7f3c769da640 1 -- 172.21.15.53:0/2577732558 shutdown_connections 2024-09-19T06:15:47.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.658+0000 7f3c769da640 1 -- 172.21.15.53:0/2577732558 wait complete. 2024-09-19T06:15:47.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.659+0000 7f3c769da640 1 Processor -- start 2024-09-19T06:15:47.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.659+0000 7f3c769da640 1 -- start start 2024-09-19T06:15:47.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.659+0000 7f3c769da640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c70109450 0x7f3c7019fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:47.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.659+0000 7f3c769da640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c70109e20 0x7f3c701a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:47.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.659+0000 7f3c769da640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3c70112f10 con 0x7f3c70109450 2024-09-19T06:15:47.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.659+0000 7f3c769da640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3c70112d90 con 0x7f3c70109e20 2024-09-19T06:15:47.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.659+0000 7f3c6ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c70109450 0x7f3c7019fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:47.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.659+0000 7f3c6ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c70109450 0x7f3c7019fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34628/0 (socket says 172.21.15.53:34628) 2024-09-19T06:15:47.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.659+0000 7f3c6ffff640 1 -- 172.21.15.53:0/3192417513 learned_addr learned my addr 172.21.15.53:0/3192417513 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:15:47.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.659+0000 7f3c6f7fe640 1 --2- 172.21.15.53:0/3192417513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c70109e20 0x7f3c701a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:47.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.659+0000 7f3c6ffff640 1 -- 172.21.15.53:0/3192417513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c70109e20 msgr2=0x7f3c701a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:47.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.659+0000 7f3c6ffff640 1 --2- 172.21.15.53:0/3192417513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c70109e20 0x7f3c701a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:47.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.660+0000 7f3c6ffff640 1 -- 172.21.15.53:0/3192417513 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c701a0720 con 0x7f3c70109450 2024-09-19T06:15:47.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.660+0000 7f3c6ffff640 1 --2- 172.21.15.53:0/3192417513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c70109450 0x7f3c7019fbf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3c6000c8f0 tx=0x7f3c6000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:47.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.660+0000 7f3c6d7fa640 1 -- 172.21.15.53:0/3192417513 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3c60004020 con 0x7f3c70109450 2024-09-19T06:15:47.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.660+0000 7f3c769da640 1 -- 172.21.15.53:0/3192417513 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c701a5190 con 0x7f3c70109450 2024-09-19T06:15:47.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.660+0000 7f3c6d7fa640 1 -- 172.21.15.53:0/3192417513 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c60007e00 con 0x7f3c70109450 2024-09-19T06:15:47.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.660+0000 7f3c6d7fa640 1 -- 172.21.15.53:0/3192417513 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3c60003d40 con 0x7f3c70109450 2024-09-19T06:15:47.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.660+0000 7f3c769da640 1 -- 172.21.15.53:0/3192417513 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c701a5770 con 0x7f3c70109450 2024-09-19T06:15:47.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.661+0000 7f3c769da640 1 -- 172.21.15.53:0/3192417513 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c70109850 con 0x7f3c70109450 2024-09-19T06:15:47.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.661+0000 7f3c6d7fa640 1 -- 172.21.15.53:0/3192417513 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f3c60007950 con 0x7f3c70109450 2024-09-19T06:15:47.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.662+0000 7f3c6d7fa640 1 --2- 172.21.15.53:0/3192417513 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f3c5806e2b0 0x7f3c58070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:47.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.662+0000 7f3c6f7fe640 1 --2- 172.21.15.53:0/3192417513 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f3c5806e2b0 0x7f3c58070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:47.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.662+0000 7f3c6d7fa640 1 -- 172.21.15.53:0/3192417513 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5255+0+0 (secure 0 0 0) 0x7f3c6008e270 con 0x7f3c70109450 2024-09-19T06:15:47.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.663+0000 7f3c6f7fe640 1 --2- 172.21.15.53:0/3192417513 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f3c5806e2b0 0x7f3c58070770 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3c701a10a0 tx=0x7f3c64004e20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:47.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.667+0000 7f3c6d7fa640 1 -- 172.21.15.53:0/3192417513 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3c60015330 con 0x7f3c70109450 2024-09-19T06:15:47.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.848+0000 7f3c769da640 1 -- 172.21.15.53:0/3192417513 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f3c70062df0 con 0x7f3c70109450 2024-09-19T06:15:47.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.849+0000 7f3c6d7fa640 1 -- 172.21.15.53:0/3192417513 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v56) ==== 74+0+130 (secure 0 0 0) 0x7f3c6005bc00 con 0x7f3c70109450 2024-09-19T06:15:47.851 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:15:47.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.851+0000 7f3c769da640 1 -- 172.21.15.53:0/3192417513 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f3c5806e2b0 msgr2=0x7f3c58070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:47.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.851+0000 7f3c769da640 1 --2- 172.21.15.53:0/3192417513 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f3c5806e2b0 0x7f3c58070770 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3c701a10a0 tx=0x7f3c64004e20 comp rx=0 tx=0).stop 2024-09-19T06:15:47.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.851+0000 7f3c769da640 1 -- 172.21.15.53:0/3192417513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c70109450 msgr2=0x7f3c7019fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:47.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.851+0000 7f3c769da640 1 --2- 172.21.15.53:0/3192417513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c70109450 0x7f3c7019fbf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3c6000c8f0 tx=0x7f3c6000cdb0 comp rx=0 tx=0).stop 2024-09-19T06:15:47.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.851+0000 7f3c769da640 1 -- 172.21.15.53:0/3192417513 shutdown_connections 2024-09-19T06:15:47.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.851+0000 7f3c769da640 1 --2- 172.21.15.53:0/3192417513 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f3c5806e2b0 0x7f3c58070770 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:47.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.851+0000 7f3c769da640 1 --2- 172.21.15.53:0/3192417513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c70109e20 0x7f3c701a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:47.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.851+0000 7f3c769da640 1 --2- 172.21.15.53:0/3192417513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c70109450 0x7f3c7019fbf0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:47.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.851+0000 7f3c769da640 1 -- 172.21.15.53:0/3192417513 >> 172.21.15.53:0/3192417513 conn(0x7f3c7006ea50 msgr2=0x7f3c7010ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:47.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.851+0000 7f3c769da640 1 -- 172.21.15.53:0/3192417513 shutdown_connections 2024-09-19T06:15:47.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:47.851+0000 7f3c769da640 1 -- 172.21.15.53:0/3192417513 wait complete. 2024-09-19T06:15:47.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:47 smithi155 ceph-mon[35292]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-09-19T06:15:47.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:47 smithi155 ceph-mon[35292]: osdmap e54: 8 total, 7 up, 8 in 2024-09-19T06:15:47.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:47 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:15:47.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:47 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:15:47.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:47 smithi155 ceph-mon[35292]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:15:47.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:47 smithi155 ceph-mon[35292]: osdmap e55: 8 total, 7 up, 8 in 2024-09-19T06:15:47.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:47 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:15:47.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:47 smithi155 ceph-mon[35292]: from='osd.7 ' entity='osd.7' 2024-09-19T06:15:47.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:47 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:47.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:47 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:47.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:47 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:15:47.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:47 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:15:47.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:47 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:15:47.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:47 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:15:47.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:47 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:47.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:47 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:15:47.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:47 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:15:47.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:47 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:48.420 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":56,"num_osds":8,"num_up_osds":8,"osd_up_since":1726726547,"num_in_osds":8,"osd_in_since":1726726526,"num_remapped_pgs":1} 2024-09-19T06:15:48.421 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:15:48.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:48 smithi053 ceph-mon[27968]: purged_snaps scrub starts 2024-09-19T06:15:48.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:48 smithi053 ceph-mon[27968]: purged_snaps scrub ok 2024-09-19T06:15:48.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:48 smithi053 ceph-mon[27968]: Detected new or changed devices on smithi155 2024-09-19T06:15:48.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:48 smithi053 ceph-mon[27968]: Adjusting osd_memory_target on smithi155 to 3728M 2024-09-19T06:15:48.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:48 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:15:48.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:48 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:15:48.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:48 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:15:48.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:48 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:15:48.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:48 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:48.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:48 smithi053 ceph-mon[27968]: osd.7 [v2:172.21.15.155:6824/2269161534,v1:172.21.15.155:6825/2269161534] boot 2024-09-19T06:15:48.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:48 smithi053 ceph-mon[27968]: osdmap e56: 8 total, 8 up, 8 in 2024-09-19T06:15:48.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:48 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:15:48.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:48 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/3192417513' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T06:15:48.684 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:15:48.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:48 smithi155 ceph-mon[35292]: purged_snaps scrub starts 2024-09-19T06:15:48.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:48 smithi155 ceph-mon[35292]: purged_snaps scrub ok 2024-09-19T06:15:48.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:48 smithi155 ceph-mon[35292]: Detected new or changed devices on smithi155 2024-09-19T06:15:48.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:48 smithi155 ceph-mon[35292]: Adjusting osd_memory_target on smithi155 to 3728M 2024-09-19T06:15:48.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:48 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:15:48.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:48 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:15:48.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:48 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:15:48.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:48 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:15:48.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:48 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:15:48.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:48 smithi155 ceph-mon[35292]: osd.7 [v2:172.21.15.155:6824/2269161534,v1:172.21.15.155:6825/2269161534] boot 2024-09-19T06:15:48.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:48 smithi155 ceph-mon[35292]: osdmap e56: 8 total, 8 up, 8 in 2024-09-19T06:15:48.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:48 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:15:48.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:48 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/3192417513' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T06:15:49.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.123+0000 7f2a1cff4640 1 -- 172.21.15.53:0/2181630983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a18101900 msgr2=0x7f2a18101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:49.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.123+0000 7f2a1cff4640 1 --2- 172.21.15.53:0/2181630983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a18101900 0x7f2a18101d60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c009880 tx=0x7f2a0c02a4a0 comp rx=0 tx=0).stop 2024-09-19T06:15:49.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.123+0000 7f2a1cff4640 1 -- 172.21.15.53:0/2181630983 shutdown_connections 2024-09-19T06:15:49.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.123+0000 7f2a1cff4640 1 --2- 172.21.15.53:0/2181630983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a18101900 0x7f2a18101d60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:49.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.123+0000 7f2a1cff4640 1 --2- 172.21.15.53:0/2181630983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a18107900 0x7f2a18107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:49.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.123+0000 7f2a1cff4640 1 -- 172.21.15.53:0/2181630983 >> 172.21.15.53:0/2181630983 conn(0x7f2a180fd600 msgr2=0x7f2a180ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:49.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.123+0000 7f2a1cff4640 1 -- 172.21.15.53:0/2181630983 shutdown_connections 2024-09-19T06:15:49.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.123+0000 7f2a1cff4640 1 -- 172.21.15.53:0/2181630983 wait complete. 2024-09-19T06:15:49.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.124+0000 7f2a1cff4640 1 Processor -- start 2024-09-19T06:15:49.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.124+0000 7f2a1cff4640 1 -- start start 2024-09-19T06:15:49.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.124+0000 7f2a1cff4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a18101900 0x7f2a1819d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:49.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.124+0000 7f2a1cff4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a18107900 0x7f2a1819dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:49.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.124+0000 7f2a1cff4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1810ed40 con 0x7f2a18107900 2024-09-19T06:15:49.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.124+0000 7f2a1cff4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1810ebc0 con 0x7f2a18101900 2024-09-19T06:15:49.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.124+0000 7f2a15d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a18107900 0x7f2a1819dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:49.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.124+0000 7f2a15d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a18107900 0x7f2a1819dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34644/0 (socket says 172.21.15.53:34644) 2024-09-19T06:15:49.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.124+0000 7f2a15d9b640 1 -- 172.21.15.53:0/2666976157 learned_addr learned my addr 172.21.15.53:0/2666976157 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:15:49.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.124+0000 7f2a1659c640 1 --2- 172.21.15.53:0/2666976157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a18101900 0x7f2a1819d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:49.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.125+0000 7f2a15d9b640 1 -- 172.21.15.53:0/2666976157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a18101900 msgr2=0x7f2a1819d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:49.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.125+0000 7f2a15d9b640 1 --2- 172.21.15.53:0/2666976157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a18101900 0x7f2a1819d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:49.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.125+0000 7f2a15d9b640 1 -- 172.21.15.53:0/2666976157 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a18197740 con 0x7f2a18107900 2024-09-19T06:15:49.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.125+0000 7f2a15d9b640 1 --2- 172.21.15.53:0/2666976157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a18107900 0x7f2a1819dbb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c02a9c0 tx=0x7f2a0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:49.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.125+0000 7f2a077fe640 1 -- 172.21.15.53:0/2666976157 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2a0c03f070 con 0x7f2a18107900 2024-09-19T06:15:49.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.125+0000 7f2a077fe640 1 -- 172.21.15.53:0/2666976157 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a0c004ac0 con 0x7f2a18107900 2024-09-19T06:15:49.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.125+0000 7f2a1cff4640 1 -- 172.21.15.53:0/2666976157 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a181979d0 con 0x7f2a18107900 2024-09-19T06:15:49.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.125+0000 7f2a077fe640 1 -- 172.21.15.53:0/2666976157 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2a0c035030 con 0x7f2a18107900 2024-09-19T06:15:49.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.126+0000 7f2a1cff4640 1 -- 172.21.15.53:0/2666976157 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a18197f00 con 0x7f2a18107900 2024-09-19T06:15:49.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.127+0000 7f2a1cff4640 1 -- 172.21.15.53:0/2666976157 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a18103040 con 0x7f2a18107900 2024-09-19T06:15:49.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.127+0000 7f2a077fe640 1 -- 172.21.15.53:0/2666976157 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f2a0c042070 con 0x7f2a18107900 2024-09-19T06:15:49.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.128+0000 7f2a077fe640 1 --2- 172.21.15.53:0/2666976157 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f2a0006e310 0x7f2a000707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:49.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.128+0000 7f2a077fe640 1 -- 172.21.15.53:0/2666976157 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5531+0+0 (secure 0 0 0) 0x7f2a0c0aeb80 con 0x7f2a18107900 2024-09-19T06:15:49.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.128+0000 7f2a1659c640 1 --2- 172.21.15.53:0/2666976157 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f2a0006e310 0x7f2a000707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:49.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.128+0000 7f2a1659c640 1 --2- 172.21.15.53:0/2666976157 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f2a0006e310 0x7f2a000707d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2a08005fd0 tx=0x7f2a08009680 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:49.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.132+0000 7f2a077fe640 1 -- 172.21.15.53:0/2666976157 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2a0c005490 con 0x7f2a18107900 2024-09-19T06:15:49.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.310+0000 7f2a1cff4640 1 -- 172.21.15.53:0/2666976157 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a18198c70 con 0x7f2a18107900 2024-09-19T06:15:49.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.311+0000 7f2a077fe640 1 -- 172.21.15.53:0/2666976157 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+13925 (secure 0 0 0) 0x7f2a0c080140 con 0x7f2a18107900 2024-09-19T06:15:49.313 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:15:49.313 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":57,"fsid":"9f2876d0-764d-11ef-bceb-c7b262605968","created":"2024-09-19T06:09:26.797853+0000","modified":"2024-09-19T06:15:48.811173+0000","last_up_change":"2024-09-19T06:15:47.809832+0000","last_in_change":"2024-09-19T06:15:26.420480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T06:13:39.241335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"35349d4a-0bde-46f6-8d5f-a019f0f89095","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1456825097},{"type":"v1","addr":"172.21.15.53:6803","nonce":1456825097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1456825097},{"type":"v1","addr":"172.21.15.53:6805","nonce":1456825097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1456825097},{"type":"v1","addr":"172.21.15.53:6809","nonce":1456825097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1456825097},{"type":"v1","addr":"172.21.15.53:6807","nonce":1456825097}]},"public_addr":"172.21.15.53:6803/1456825097","cluster_addr":"172.21.15.53:6805/1456825097","heartbeat_back_addr":"172.21.15.53:6809/1456825097","heartbeat_front_addr":"172.21.15.53:6807/1456825097","state":["exists","up"]},{"osd":1,"uuid":"832c2772-a460-4691-8db7-ad1d0c48cc9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3638097004},{"type":"v1","addr":"172.21.15.53:6811","nonce":3638097004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3638097004},{"type":"v1","addr":"172.21.15.53:6813","nonce":3638097004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3638097004},{"type":"v1","addr":"172.21.15.53:6817","nonce":3638097004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3638097004},{"type":"v1","addr":"172.21.15.53:6815","nonce":3638097004}]},"public_addr":"172.21.15.53:6811/3638097004","cluster_addr":"172.21.15.53:6813/3638097004","heartbeat_back_addr":"172.21.15.53:6817/3638097004","heartbeat_front_addr":"172.21.15.53:6815/3638097004","state":["exists","up"]},{"osd":2,"uuid":"57282117-559f-4495-b85f-90b68b2439ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":1988602192},{"type":"v1","addr":"172.21.15.53:6819","nonce":1988602192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":1988602192},{"type":"v1","addr":"172.21.15.53:6821","nonce":1988602192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1988602192},{"type":"v1","addr":"172.21.15.53:6825","nonce":1988602192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":1988602192},{"type":"v1","addr":"172.21.15.53:6823","nonce":1988602192}]},"public_addr":"172.21.15.53:6819/1988602192","cluster_addr":"172.21.15.53:6821/1988602192","heartbeat_back_addr":"172.21.15.53:6825/1988602192","heartbeat_front_addr":"172.21.15.53:6823/1988602192","state":["exists","up"]},{"osd":3,"uuid":"e6bf8cf0-2419-4c2a-b562-9221a117050c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2604012854},{"type":"v1","addr":"172.21.15.53:6827","nonce":2604012854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2604012854},{"type":"v1","addr":"172.21.15.53:6829","nonce":2604012854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2604012854},{"type":"v1","addr":"172.21.15.53:6833","nonce":2604012854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2604012854},{"type":"v1","addr":"172.21.15.53:6831","nonce":2604012854}]},"public_addr":"172.21.15.53:6827/2604012854","cluster_addr":"172.21.15.53:6829/2604012854","heartbeat_back_addr":"172.21.15.53:6833/2604012854","heartbeat_front_addr":"172.21.15.53:6831/2604012854","state":["exists","up"]},{"osd":4,"uuid":"7adedf0a-6fbc-42bc-8996-ba482b638929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2046029108},{"type":"v1","addr":"172.21.15.155:6801","nonce":2046029108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2046029108},{"type":"v1","addr":"172.21.15.155:6803","nonce":2046029108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2046029108},{"type":"v1","addr":"172.21.15.155:6807","nonce":2046029108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2046029108},{"type":"v1","addr":"172.21.15.155:6805","nonce":2046029108}]},"public_addr":"172.21.15.155:6801/2046029108","cluster_addr":"172.21.15.155:6803/2046029108","heartbeat_back_addr":"172.21.15.155:6807/2046029108","heartbeat_front_addr":"172.21.15.155:6805/2046029108","state":["exists","up"]},{"osd":5,"uuid":"c664e57c-20b5-46d4-8830-70780a91d6dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":464507356},{"type":"v1","addr":"172.21.15.155:6809","nonce":464507356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":464507356},{"type":"v1","addr":"172.21.15.155:6811","nonce":464507356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":464507356},{"type":"v1","addr":"172.21.15.155:6815","nonce":464507356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":464507356},{"type":"v1","addr":"172.21.15.155:6813","nonce":464507356}]},"public_addr":"172.21.15.155:6809/464507356","cluster_addr":"172.21.15.155:6811/464507356","heartbeat_back_addr":"172.21.15.155:6815/464507356","heartbeat_front_addr":"172.21.15.155:6813/464507356","state":["exists","up"]},{"osd":6,"uuid":"bc8a88f5-75ab-4482-88e0-5cc5e6f06db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2574655677},{"type":"v1","addr":"172.21.15.155:6817","nonce":2574655677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2574655677},{"type":"v1","addr":"172.21.15.155:6819","nonce":2574655677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2574655677},{"type":"v1","addr":"172.21.15.155:6823","nonce":2574655677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2574655677},{"type":"v1","addr":"172.21.15.155:6821","nonce":2574655677}]},"public_addr":"172.21.15.155:6817/2574655677","cluster_addr":"172.21.15.155:6819/2574655677","heartbeat_back_addr":"172.21.15.155:6823/2574655677","heartbeat_front_addr":"172.21.15.155:6821/2574655677","state":["exists","up"]},{"osd":7,"uuid":"b91759fe-f27c-4ab2-9bd3-b5961b358d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2269161534},{"type":"v1","addr":"172.21.15.155:6825","nonce":2269161534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2269161534},{"type":"v1","addr":"172.21.15.155:6827","nonce":2269161534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2269161534},{"type":"v1","addr":"172.21.15.155:6831","nonce":2269161534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2269161534},{"type":"v1","addr":"172.21.15.155:6829","nonce":2269161534}]},"public_addr":"172.21.15.155:6825/2269161534","cluster_addr":"172.21.15.155:6827/2269161534","heartbeat_back_addr":"172.21.15.155:6831/2269161534","heartbeat_front_addr":"172.21.15.155:6829/2269161534","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:12:43.760347+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:13:10.493898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:13:37.244856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:14:05.006717+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:14:29.801028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:14:55.095341+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:15:20.683202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:15:45.635704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"1.0","osds":[0,6]}],"primary_temp":[],"blocklist":{"172.21.15.53:6801/3684090301":"2024-09-20T06:11:28.721009+0000","172.21.15.53:6800/475342876":"2024-09-20T06:09:58.271775+0000","172.21.15.53:6801/475342876":"2024-09-20T06:09:58.271775+0000","172.21.15.53:0/3753705881":"2024-09-20T06:11:28.721009+0000","172.21.15.53:0/4216592427":"2024-09-20T06:09:58.271775+0000","172.21.15.53:0/1618433917":"2024-09-20T06:10:30.209033+0000","172.21.15.53:0/623598636":"2024-09-20T06:09:58.271775+0000","172.21.15.53:6800/3684090301":"2024-09-20T06:11:28.721009+0000","172.21.15.53:6800/681468213":"2024-09-20T06:10:30.209033+0000","172.21.15.53:0/597749918":"2024-09-20T06:10:30.209033+0000","172.21.15.53:0/21853845":"2024-09-20T06:11:28.721009+0000","172.21.15.53:0/3515303955":"2024-09-20T06:09:58.271775+0000","172.21.15.53:6801/681468213":"2024-09-20T06:10:30.209033+0000","172.21.15.53:0/1729980812":"2024-09-20T06:11:28.721009+0000","172.21.15.53:0/372276981":"2024-09-20T06:10:30.209033+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:15:49.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.314+0000 7f2a1cff4640 1 -- 172.21.15.53:0/2666976157 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f2a0006e310 msgr2=0x7f2a000707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:49.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.314+0000 7f2a1cff4640 1 --2- 172.21.15.53:0/2666976157 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f2a0006e310 0x7f2a000707d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2a08005fd0 tx=0x7f2a08009680 comp rx=0 tx=0).stop 2024-09-19T06:15:49.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.314+0000 7f2a1cff4640 1 -- 172.21.15.53:0/2666976157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a18107900 msgr2=0x7f2a1819dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:49.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.314+0000 7f2a1cff4640 1 --2- 172.21.15.53:0/2666976157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a18107900 0x7f2a1819dbb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c02a9c0 tx=0x7f2a0c0046c0 comp rx=0 tx=0).stop 2024-09-19T06:15:49.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.314+0000 7f2a1cff4640 1 -- 172.21.15.53:0/2666976157 shutdown_connections 2024-09-19T06:15:49.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.314+0000 7f2a1cff4640 1 --2- 172.21.15.53:0/2666976157 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f2a0006e310 0x7f2a000707d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:49.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.314+0000 7f2a1cff4640 1 --2- 172.21.15.53:0/2666976157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a18107900 0x7f2a1819dbb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:49.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.314+0000 7f2a1cff4640 1 --2- 172.21.15.53:0/2666976157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a18101900 0x7f2a1819d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:49.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.314+0000 7f2a1cff4640 1 -- 172.21.15.53:0/2666976157 >> 172.21.15.53:0/2666976157 conn(0x7f2a180fd600 msgr2=0x7f2a180fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:49.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.314+0000 7f2a1cff4640 1 -- 172.21.15.53:0/2666976157 shutdown_connections 2024-09-19T06:15:49.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:49.314+0000 7f2a1cff4640 1 -- 172.21.15.53:0/2666976157 wait complete. 2024-09-19T06:15:49.856 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:49 smithi053 ceph-mon[27968]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:15:49.857 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:49 smithi053 ceph-mon[27968]: osdmap e57: 8 total, 8 up, 8 in 2024-09-19T06:15:49.857 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:49 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/2666976157' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:15:49.977 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T06:13:39.241335+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T06:15:49.977 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-19T06:15:49.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:49 smithi155 ceph-mon[35292]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:15:49.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:49 smithi155 ceph-mon[35292]: osdmap e57: 8 total, 8 up, 8 in 2024-09-19T06:15:49.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:49 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/2666976157' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:15:50.241 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:15:50.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.693+0000 7f9be6da7640 1 -- 172.21.15.53:0/1870991090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9be0107900 msgr2=0x7f9be0107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:50.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.693+0000 7f9be6da7640 1 --2- 172.21.15.53:0/1870991090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9be0107900 0x7f9be0107ce0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009880 tx=0x7f9bd402a4a0 comp rx=0 tx=0).stop 2024-09-19T06:15:50.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.693+0000 7f9be6da7640 1 -- 172.21.15.53:0/1870991090 shutdown_connections 2024-09-19T06:15:50.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.693+0000 7f9be6da7640 1 --2- 172.21.15.53:0/1870991090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9be0101900 0x7f9be0101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:50.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.693+0000 7f9be6da7640 1 --2- 172.21.15.53:0/1870991090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9be0107900 0x7f9be0107ce0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:50.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.693+0000 7f9be6da7640 1 -- 172.21.15.53:0/1870991090 >> 172.21.15.53:0/1870991090 conn(0x7f9be00fd600 msgr2=0x7f9be00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:50.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.693+0000 7f9be6da7640 1 -- 172.21.15.53:0/1870991090 shutdown_connections 2024-09-19T06:15:50.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.693+0000 7f9be6da7640 1 -- 172.21.15.53:0/1870991090 wait complete. 2024-09-19T06:15:50.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.694+0000 7f9be6da7640 1 Processor -- start 2024-09-19T06:15:50.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.694+0000 7f9be6da7640 1 -- start start 2024-09-19T06:15:50.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.694+0000 7f9be6da7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9be0101900 0x7f9be019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:50.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.694+0000 7f9be6da7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9be0107900 0x7f9be019db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:50.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.694+0000 7f9be6da7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9be010ed20 con 0x7f9be0101900 2024-09-19T06:15:50.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.694+0000 7f9be6da7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9be010eba0 con 0x7f9be0107900 2024-09-19T06:15:50.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.694+0000 7f9be4b43640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9be0101900 0x7f9be019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:50.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.694+0000 7f9be4b43640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9be0101900 0x7f9be019d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34662/0 (socket says 172.21.15.53:34662) 2024-09-19T06:15:50.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.694+0000 7f9be4b43640 1 -- 172.21.15.53:0/202398529 learned_addr learned my addr 172.21.15.53:0/202398529 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:15:50.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.694+0000 7f9bdffff640 1 --2- 172.21.15.53:0/202398529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9be0107900 0x7f9be019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:50.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.695+0000 7f9be4b43640 1 -- 172.21.15.53:0/202398529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9be0107900 msgr2=0x7f9be019db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:50.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.695+0000 7f9be4b43640 1 --2- 172.21.15.53:0/202398529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9be0107900 0x7f9be019db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:50.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.695+0000 7f9be4b43640 1 -- 172.21.15.53:0/202398529 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be0197710 con 0x7f9be0101900 2024-09-19T06:15:50.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.695+0000 7f9be4b43640 1 --2- 172.21.15.53:0/202398529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9be0101900 0x7f9be019d640 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9bd402a9c0 tx=0x7f9bd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:50.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.695+0000 7f9bddffb640 1 -- 172.21.15.53:0/202398529 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9bd403f070 con 0x7f9be0101900 2024-09-19T06:15:50.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.695+0000 7f9bddffb640 1 -- 172.21.15.53:0/202398529 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bd4004ac0 con 0x7f9be0101900 2024-09-19T06:15:50.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.695+0000 7f9be6da7640 1 -- 172.21.15.53:0/202398529 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9be01979a0 con 0x7f9be0101900 2024-09-19T06:15:50.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.695+0000 7f9bddffb640 1 -- 172.21.15.53:0/202398529 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9bd4035030 con 0x7f9be0101900 2024-09-19T06:15:50.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.695+0000 7f9be6da7640 1 -- 172.21.15.53:0/202398529 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9be0197ed0 con 0x7f9be0101900 2024-09-19T06:15:50.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.696+0000 7f9be6da7640 1 -- 172.21.15.53:0/202398529 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9be0103040 con 0x7f9be0101900 2024-09-19T06:15:50.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.697+0000 7f9bddffb640 1 -- 172.21.15.53:0/202398529 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f9bd4042070 con 0x7f9be0101900 2024-09-19T06:15:50.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.697+0000 7f9bddffb640 1 --2- 172.21.15.53:0/202398529 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f9bd006e2b0 0x7f9bd0070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:50.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.697+0000 7f9bddffb640 1 -- 172.21.15.53:0/202398529 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5531+0+0 (secure 0 0 0) 0x7f9bd40aed40 con 0x7f9be0101900 2024-09-19T06:15:50.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.697+0000 7f9bdffff640 1 --2- 172.21.15.53:0/202398529 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f9bd006e2b0 0x7f9bd0070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:50.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.698+0000 7f9bdffff640 1 --2- 172.21.15.53:0/202398529 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f9bd006e2b0 0x7f9bd0070770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9be0198f60 tx=0x7f9bcc00b590 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:50.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.702+0000 7f9bddffb640 1 -- 172.21.15.53:0/202398529 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9bd407c5d0 con 0x7f9be0101900 2024-09-19T06:15:50.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.879+0000 7f9be6da7640 1 -- 172.21.15.53:0/202398529 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9be0104b10 con 0x7f9be0101900 2024-09-19T06:15:50.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.880+0000 7f9bddffb640 1 -- 172.21.15.53:0/202398529 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) ==== 93+0+10 (secure 0 0 0) 0x7f9bd4080280 con 0x7f9be0101900 2024-09-19T06:15:50.882 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2024-09-19T06:15:50.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.882+0000 7f9be6da7640 1 -- 172.21.15.53:0/202398529 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f9bd006e2b0 msgr2=0x7f9bd0070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:50.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.882+0000 7f9be6da7640 1 --2- 172.21.15.53:0/202398529 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f9bd006e2b0 0x7f9bd0070770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9be0198f60 tx=0x7f9bcc00b590 comp rx=0 tx=0).stop 2024-09-19T06:15:50.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.882+0000 7f9be6da7640 1 -- 172.21.15.53:0/202398529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9be0101900 msgr2=0x7f9be019d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:50.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.882+0000 7f9be6da7640 1 --2- 172.21.15.53:0/202398529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9be0101900 0x7f9be019d640 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9bd402a9c0 tx=0x7f9bd40046c0 comp rx=0 tx=0).stop 2024-09-19T06:15:50.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.882+0000 7f9be6da7640 1 -- 172.21.15.53:0/202398529 shutdown_connections 2024-09-19T06:15:50.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.882+0000 7f9be6da7640 1 --2- 172.21.15.53:0/202398529 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f9bd006e2b0 0x7f9bd0070770 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:50.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.882+0000 7f9be6da7640 1 --2- 172.21.15.53:0/202398529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9be0107900 0x7f9be019db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:50.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.882+0000 7f9be6da7640 1 --2- 172.21.15.53:0/202398529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9be0101900 0x7f9be019d640 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:50.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.882+0000 7f9be6da7640 1 -- 172.21.15.53:0/202398529 >> 172.21.15.53:0/202398529 conn(0x7f9be00fd600 msgr2=0x7f9be00ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:50.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.882+0000 7f9be6da7640 1 -- 172.21.15.53:0/202398529 shutdown_connections 2024-09-19T06:15:50.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:50.882+0000 7f9be6da7640 1 -- 172.21.15.53:0/202398529 wait complete. 2024-09-19T06:15:50.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:50 smithi155 ceph-mon[35292]: osdmap e58: 8 total, 8 up, 8 in 2024-09-19T06:15:50.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:50 smithi155 ceph-mon[35292]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:15:51.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:50 smithi053 ceph-mon[27968]: osdmap e58: 8 total, 8 up, 8 in 2024-09-19T06:15:51.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:50 smithi053 ceph-mon[27968]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:15:51.725 INFO:tasks.cephadm:Setting up client nodes... 2024-09-19T06:15:51.726 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-19T06:15:51.995 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:15:52.109 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:51 smithi053 ceph-mon[27968]: osdmap e59: 8 total, 8 up, 8 in 2024-09-19T06:15:52.109 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:51 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/202398529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T06:15:52.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:51 smithi155 ceph-mon[35292]: osdmap e59: 8 total, 8 up, 8 in 2024-09-19T06:15:52.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:51 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/202398529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T06:15:52.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.556+0000 7f03b463f640 1 -- 172.21.15.53:0/2694085401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03ac104d50 msgr2=0x7f03ac107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:52.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.556+0000 7f03b463f640 1 --2- 172.21.15.53:0/2694085401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03ac104d50 0x7f03ac107140 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f03a0009880 tx=0x7f03a002a4a0 comp rx=0 tx=0).stop 2024-09-19T06:15:52.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.556+0000 7f03b463f640 1 -- 172.21.15.53:0/2694085401 shutdown_connections 2024-09-19T06:15:52.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.556+0000 7f03b463f640 1 --2- 172.21.15.53:0/2694085401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03ac104d50 0x7f03ac107140 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:52.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.556+0000 7f03b463f640 1 --2- 172.21.15.53:0/2694085401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03ac102420 0x7f03ac104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:52.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.556+0000 7f03b463f640 1 -- 172.21.15.53:0/2694085401 >> 172.21.15.53:0/2694085401 conn(0x7f03ac0fbfa0 msgr2=0x7f03ac0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:52.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.556+0000 7f03b463f640 1 -- 172.21.15.53:0/2694085401 shutdown_connections 2024-09-19T06:15:52.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.557+0000 7f03b463f640 1 -- 172.21.15.53:0/2694085401 wait complete. 2024-09-19T06:15:52.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.557+0000 7f03b463f640 1 Processor -- start 2024-09-19T06:15:52.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.557+0000 7f03b463f640 1 -- start start 2024-09-19T06:15:52.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.558+0000 7f03b463f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03ac102420 0x7f03ac1973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:52.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.558+0000 7f03b463f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03ac104d50 0x7f03ac197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:52.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.558+0000 7f03b463f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac10a680 con 0x7f03ac104d50 2024-09-19T06:15:52.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.558+0000 7f03b463f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac10a500 con 0x7f03ac102420 2024-09-19T06:15:52.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.558+0000 7f03b1bda640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03ac104d50 0x7f03ac197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:52.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.558+0000 7f03b1bda640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03ac104d50 0x7f03ac197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33998/0 (socket says 172.21.15.53:33998) 2024-09-19T06:15:52.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.558+0000 7f03b23db640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03ac102420 0x7f03ac1973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:52.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.558+0000 7f03b1bda640 1 -- 172.21.15.53:0/1954013335 learned_addr learned my addr 172.21.15.53:0/1954013335 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:15:52.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.558+0000 7f03b1bda640 1 -- 172.21.15.53:0/1954013335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03ac102420 msgr2=0x7f03ac1973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:52.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.558+0000 7f03b1bda640 1 --2- 172.21.15.53:0/1954013335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03ac102420 0x7f03ac1973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:52.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.558+0000 7f03b1bda640 1 -- 172.21.15.53:0/1954013335 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ac197f00 con 0x7f03ac104d50 2024-09-19T06:15:52.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.558+0000 7f03b1bda640 1 --2- 172.21.15.53:0/1954013335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03ac104d50 0x7f03ac197910 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f03a0006010 tx=0x7f03a00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:52.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.559+0000 7f03ab7fe640 1 -- 172.21.15.53:0/1954013335 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f03a003f070 con 0x7f03ac104d50 2024-09-19T06:15:52.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.559+0000 7f03b463f640 1 -- 172.21.15.53:0/1954013335 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03ac19c970 con 0x7f03ac104d50 2024-09-19T06:15:52.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.559+0000 7f03ab7fe640 1 -- 172.21.15.53:0/1954013335 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03a00049d0 con 0x7f03ac104d50 2024-09-19T06:15:52.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.559+0000 7f03ab7fe640 1 -- 172.21.15.53:0/1954013335 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f03a0035030 con 0x7f03ac104d50 2024-09-19T06:15:52.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.559+0000 7f03b463f640 1 -- 172.21.15.53:0/1954013335 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03ac19cf50 con 0x7f03ac104d50 2024-09-19T06:15:52.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.560+0000 7f03b463f640 1 -- 172.21.15.53:0/1954013335 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03ac102800 con 0x7f03ac104d50 2024-09-19T06:15:52.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.560+0000 7f03ab7fe640 1 -- 172.21.15.53:0/1954013335 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f03a0042070 con 0x7f03ac104d50 2024-09-19T06:15:52.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.561+0000 7f03ab7fe640 1 --2- 172.21.15.53:0/1954013335 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f039406e2b0 0x7f0394070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:52.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.561+0000 7f03ab7fe640 1 -- 172.21.15.53:0/1954013335 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5502+0+0 (secure 0 0 0) 0x7f03a00aed50 con 0x7f03ac104d50 2024-09-19T06:15:52.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.561+0000 7f03b23db640 1 --2- 172.21.15.53:0/1954013335 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f039406e2b0 0x7f0394070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:52.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.561+0000 7f03b23db640 1 --2- 172.21.15.53:0/1954013335 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f039406e2b0 0x7f0394070770 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f03a4005fd0 tx=0x7f03a400b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:52.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.566+0000 7f03ab7fe640 1 -- 172.21.15.53:0/1954013335 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f03a00802b0 con 0x7f03ac104d50 2024-09-19T06:15:52.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.815+0000 7f03b463f640 1 -- 172.21.15.53:0/1954013335 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f03ac19d380 con 0x7f03ac104d50 2024-09-19T06:15:52.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.817+0000 7f03ab7fe640 1 -- 172.21.15.53:0/1954013335 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v18) ==== 170+0+59 (secure 0 0 0) 0x7f03a0084ae0 con 0x7f03ac104d50 2024-09-19T06:15:52.818 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2024-09-19T06:15:52.819 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQCYwetmOferMBAAHD+EgPYAw4FW7NbtlLiufQ== 2024-09-19T06:15:52.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.819+0000 7f03b463f640 1 -- 172.21.15.53:0/1954013335 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f039406e2b0 msgr2=0x7f0394070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:52.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.819+0000 7f03b463f640 1 --2- 172.21.15.53:0/1954013335 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f039406e2b0 0x7f0394070770 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f03a4005fd0 tx=0x7f03a400b390 comp rx=0 tx=0).stop 2024-09-19T06:15:52.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.819+0000 7f03b463f640 1 -- 172.21.15.53:0/1954013335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03ac104d50 msgr2=0x7f03ac197910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:52.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.819+0000 7f03b463f640 1 --2- 172.21.15.53:0/1954013335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03ac104d50 0x7f03ac197910 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f03a0006010 tx=0x7f03a00045e0 comp rx=0 tx=0).stop 2024-09-19T06:15:52.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.819+0000 7f03b463f640 1 -- 172.21.15.53:0/1954013335 shutdown_connections 2024-09-19T06:15:52.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.819+0000 7f03b463f640 1 --2- 172.21.15.53:0/1954013335 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f039406e2b0 0x7f0394070770 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:52.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.819+0000 7f03b463f640 1 --2- 172.21.15.53:0/1954013335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03ac104d50 0x7f03ac197910 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:52.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.819+0000 7f03b463f640 1 --2- 172.21.15.53:0/1954013335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03ac102420 0x7f03ac1973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:52.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.819+0000 7f03b463f640 1 -- 172.21.15.53:0/1954013335 >> 172.21.15.53:0/1954013335 conn(0x7f03ac0fbfa0 msgr2=0x7f03ac0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:52.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.820+0000 7f03b463f640 1 -- 172.21.15.53:0/1954013335 shutdown_connections 2024-09-19T06:15:52.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:52.820+0000 7f03b463f640 1 -- 172.21.15.53:0/1954013335 wait complete. 2024-09-19T06:15:53.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:52 smithi053 ceph-mon[27968]: osdmap e60: 8 total, 8 up, 8 in 2024-09-19T06:15:53.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:52 smithi053 ceph-mon[27968]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:15:53.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:52 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/1954013335' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T06:15:53.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:52 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/1954013335' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T06:15:53.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:52 smithi155 ceph-mon[35292]: osdmap e60: 8 total, 8 up, 8 in 2024-09-19T06:15:53.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:52 smithi155 ceph-mon[35292]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:15:53.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:52 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/1954013335' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T06:15:53.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:52 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/1954013335' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T06:15:53.374 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-19T06:15:53.374 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-19T06:15:53.374 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-19T06:15:53.412 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-19T06:15:53.672 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi155/config 2024-09-19T06:15:54.154 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.146+0000 7fe6f72fb640 1 -- 172.21.15.155:0/3840463502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6f0104d20 msgr2=0x7fe6f0107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:54.154 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.146+0000 7fe6f72fb640 1 --2- 172.21.15.155:0/3840463502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6f0104d20 0x7fe6f0107110 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0009880 tx=0x7fe6e002a4a0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.154 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.146+0000 7fe6f72fb640 1 -- 172.21.15.155:0/3840463502 shutdown_connections 2024-09-19T06:15:54.154 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.146+0000 7fe6f72fb640 1 --2- 172.21.15.155:0/3840463502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6f0104d20 0x7fe6f0107110 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.154 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.146+0000 7fe6f72fb640 1 --2- 172.21.15.155:0/3840463502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6f01023f0 0x7fe6f01047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.154 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.146+0000 7fe6f72fb640 1 -- 172.21.15.155:0/3840463502 >> 172.21.15.155:0/3840463502 conn(0x7fe6f00fc010 msgr2=0x7fe6f00fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:54.154 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.146+0000 7fe6f72fb640 1 -- 172.21.15.155:0/3840463502 shutdown_connections 2024-09-19T06:15:54.154 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.146+0000 7fe6f72fb640 1 -- 172.21.15.155:0/3840463502 wait complete. 2024-09-19T06:15:54.154 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.147+0000 7fe6f72fb640 1 Processor -- start 2024-09-19T06:15:54.154 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.147+0000 7fe6f72fb640 1 -- start start 2024-09-19T06:15:54.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.147+0000 7fe6f72fb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6f01023f0 0x7fe6f0197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:54.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.147+0000 7fe6f72fb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6f0104d20 0x7fe6f0197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:54.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.147+0000 7fe6f72fb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f010a720 con 0x7fe6f0104d20 2024-09-19T06:15:54.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.147+0000 7fe6f72fb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f010a5a0 con 0x7fe6f01023f0 2024-09-19T06:15:54.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.147+0000 7fe6f5097640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6f01023f0 0x7fe6f0197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:54.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.147+0000 7fe6f5097640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6f01023f0 0x7fe6f0197450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:47906/0 (socket says 172.21.15.155:47906) 2024-09-19T06:15:54.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.147+0000 7fe6f5097640 1 -- 172.21.15.155:0/1063685358 learned_addr learned my addr 172.21.15.155:0/1063685358 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-19T06:15:54.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.147+0000 7fe6f4896640 1 --2- 172.21.15.155:0/1063685358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6f0104d20 0x7fe6f0197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:54.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.147+0000 7fe6f5097640 1 -- 172.21.15.155:0/1063685358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6f0104d20 msgr2=0x7fe6f0197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:54.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.147+0000 7fe6f5097640 1 --2- 172.21.15.155:0/1063685358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6f0104d20 0x7fe6f0197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.147+0000 7fe6f5097640 1 -- 172.21.15.155:0/1063685358 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6f0197f80 con 0x7fe6f01023f0 2024-09-19T06:15:54.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.148+0000 7fe6f5097640 1 --2- 172.21.15.155:0/1063685358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6f01023f0 0x7fe6f0197450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec00d6f0 tx=0x7fe6ec00dbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:54.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.148+0000 7fe6e67fc640 1 -- 172.21.15.155:0/1063685358 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe6ec017070 con 0x7fe6f01023f0 2024-09-19T06:15:54.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.148+0000 7fe6e67fc640 1 -- 172.21.15.155:0/1063685358 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6ec00de00 con 0x7fe6f01023f0 2024-09-19T06:15:54.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.148+0000 7fe6f72fb640 1 -- 172.21.15.155:0/1063685358 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6f019c9f0 con 0x7fe6f01023f0 2024-09-19T06:15:54.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.148+0000 7fe6e67fc640 1 -- 172.21.15.155:0/1063685358 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe6ec004aa0 con 0x7fe6f01023f0 2024-09-19T06:15:54.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.148+0000 7fe6f72fb640 1 -- 172.21.15.155:0/1063685358 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6f019cfd0 con 0x7fe6f01023f0 2024-09-19T06:15:54.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.149+0000 7fe6f72fb640 1 -- 172.21.15.155:0/1063685358 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6f010aea0 con 0x7fe6f01023f0 2024-09-19T06:15:54.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.149+0000 7fe6e67fc640 1 -- 172.21.15.155:0/1063685358 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7fe6ec004d50 con 0x7fe6f01023f0 2024-09-19T06:15:54.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.150+0000 7fe6e67fc640 1 --2- 172.21.15.155:0/1063685358 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fe6dc06e2b0 0x7fe6dc070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:54.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.150+0000 7fe6e67fc640 1 -- 172.21.15.155:0/1063685358 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5502+0+0 (secure 0 0 0) 0x7fe6ec08de50 con 0x7fe6f01023f0 2024-09-19T06:15:54.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.150+0000 7fe6f4896640 1 --2- 172.21.15.155:0/1063685358 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fe6dc06e2b0 0x7fe6dc070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:54.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.150+0000 7fe6f4896640 1 --2- 172.21.15.155:0/1063685358 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fe6dc06e2b0 0x7fe6dc070770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0006010 tx=0x7fe6e00095f0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:54.157 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.154+0000 7fe6e67fc640 1 -- 172.21.15.155:0/1063685358 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe6f010aea0 con 0x7fe6f01023f0 2024-09-19T06:15:54.389 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.388+0000 7fe6f72fb640 1 -- 172.21.15.155:0/1063685358 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fe6f019d8b0 con 0x7fe6f01023f0 2024-09-19T06:15:54.394 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.393+0000 7fe6e67fc640 1 -- 172.21.15.155:0/1063685358 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v19) ==== 170+0+59 (secure 0 0 0) 0x7fe6ec05b700 con 0x7fe6f01023f0 2024-09-19T06:15:54.394 INFO:teuthology.orchestra.run.smithi155.stdout:[client.1] 2024-09-19T06:15:54.394 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCawetmQK1DFxAAc6O42xVc+WuOkfFyWnaxWw== 2024-09-19T06:15:54.396 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.395+0000 7fe6f72fb640 1 -- 172.21.15.155:0/1063685358 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fe6dc06e2b0 msgr2=0x7fe6dc070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:54.396 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.395+0000 7fe6f72fb640 1 --2- 172.21.15.155:0/1063685358 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fe6dc06e2b0 0x7fe6dc070770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0006010 tx=0x7fe6e00095f0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.395+0000 7fe6f72fb640 1 -- 172.21.15.155:0/1063685358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6f01023f0 msgr2=0x7fe6f0197450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:54.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.395+0000 7fe6f72fb640 1 --2- 172.21.15.155:0/1063685358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6f01023f0 0x7fe6f0197450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec00d6f0 tx=0x7fe6ec00dbb0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.395+0000 7fe6f72fb640 1 -- 172.21.15.155:0/1063685358 shutdown_connections 2024-09-19T06:15:54.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.395+0000 7fe6f72fb640 1 --2- 172.21.15.155:0/1063685358 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fe6dc06e2b0 0x7fe6dc070770 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.395+0000 7fe6f72fb640 1 --2- 172.21.15.155:0/1063685358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6f0104d20 0x7fe6f0197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.395+0000 7fe6f72fb640 1 --2- 172.21.15.155:0/1063685358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6f01023f0 0x7fe6f0197450 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.395+0000 7fe6f72fb640 1 -- 172.21.15.155:0/1063685358 >> 172.21.15.155:0/1063685358 conn(0x7fe6f00fc010 msgr2=0x7fe6f00fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:54.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.395+0000 7fe6f72fb640 1 -- 172.21.15.155:0/1063685358 shutdown_connections 2024-09-19T06:15:54.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:15:54.395+0000 7fe6f72fb640 1 -- 172.21.15.155:0/1063685358 wait complete. 2024-09-19T06:15:54.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:54 smithi155 ceph-mon[35292]: from='client.? 172.21.15.155:0/1063685358' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T06:15:54.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:54 smithi155 ceph-mon[35292]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T06:15:54.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:54 smithi155 ceph-mon[35292]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T06:15:54.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:54 smithi053 ceph-mon[27968]: from='client.? 172.21.15.155:0/1063685358' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T06:15:54.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:54 smithi053 ceph-mon[27968]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T06:15:54.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:54 smithi053 ceph-mon[27968]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T06:15:55.026 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-19T06:15:55.026 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-19T06:15:55.026 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-19T06:15:55.065 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-19T06:15:55.066 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-19T06:15:55.066 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-19T06:15:55.338 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:15:55.607 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:55 smithi053 ceph-mon[27968]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-19T06:15:55.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:55 smithi155 ceph-mon[35292]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-19T06:15:55.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.825+0000 7fa91366c640 1 -- 172.21.15.53:0/74246413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa90c102ab0 msgr2=0x7fa90c102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:55.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.825+0000 7fa91366c640 1 --2- 172.21.15.53:0/74246413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa90c102ab0 0x7fa90c102e90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa908009880 tx=0x7fa90802a530 comp rx=0 tx=0).stop 2024-09-19T06:15:55.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.825+0000 7fa91366c640 1 -- 172.21.15.53:0/74246413 shutdown_connections 2024-09-19T06:15:55.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.825+0000 7fa91366c640 1 --2- 172.21.15.53:0/74246413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa90c1033d0 0x7fa90c10b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:55.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.825+0000 7fa91366c640 1 --2- 172.21.15.53:0/74246413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa90c102ab0 0x7fa90c102e90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:55.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.825+0000 7fa91366c640 1 -- 172.21.15.53:0/74246413 >> 172.21.15.53:0/74246413 conn(0x7fa90c0fc550 msgr2=0x7fa90c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:55.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.825+0000 7fa91366c640 1 -- 172.21.15.53:0/74246413 shutdown_connections 2024-09-19T06:15:55.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.826+0000 7fa91366c640 1 -- 172.21.15.53:0/74246413 wait complete. 2024-09-19T06:15:55.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.826+0000 7fa91366c640 1 Processor -- start 2024-09-19T06:15:55.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.826+0000 7fa91366c640 1 -- start start 2024-09-19T06:15:55.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.827+0000 7fa91366c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa90c102ab0 0x7fa90c100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:55.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.827+0000 7fa91366c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa90c1033d0 0x7fa90c100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:55.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.827+0000 7fa91366c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c10ed30 con 0x7fa90c1033d0 2024-09-19T06:15:55.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.827+0000 7fa91366c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c10ebb0 con 0x7fa90c102ab0 2024-09-19T06:15:55.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.827+0000 7fa910c07640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa90c1033d0 0x7fa90c100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:55.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.827+0000 7fa910c07640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa90c1033d0 0x7fa90c100c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34012/0 (socket says 172.21.15.53:34012) 2024-09-19T06:15:55.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.827+0000 7fa910c07640 1 -- 172.21.15.53:0/3040404947 learned_addr learned my addr 172.21.15.53:0/3040404947 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:15:55.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.827+0000 7fa911408640 1 --2- 172.21.15.53:0/3040404947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa90c102ab0 0x7fa90c100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:55.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.827+0000 7fa910c07640 1 -- 172.21.15.53:0/3040404947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa90c102ab0 msgr2=0x7fa90c100710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:55.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.827+0000 7fa910c07640 1 --2- 172.21.15.53:0/3040404947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa90c102ab0 0x7fa90c100710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:55.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.827+0000 7fa910c07640 1 -- 172.21.15.53:0/3040404947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa90c101190 con 0x7fa90c1033d0 2024-09-19T06:15:55.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.827+0000 7fa910c07640 1 --2- 172.21.15.53:0/3040404947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa90c1033d0 0x7fa90c100c50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa90000c8f0 tx=0x7fa90000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:55.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.828+0000 7fa8fe7fc640 1 -- 172.21.15.53:0/3040404947 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa9000079b0 con 0x7fa90c1033d0 2024-09-19T06:15:55.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.828+0000 7fa8fe7fc640 1 -- 172.21.15.53:0/3040404947 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa90000f450 con 0x7fa90c1033d0 2024-09-19T06:15:55.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.828+0000 7fa91366c640 1 -- 172.21.15.53:0/3040404947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa90c101480 con 0x7fa90c1033d0 2024-09-19T06:15:55.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.828+0000 7fa8fe7fc640 1 -- 172.21.15.53:0/3040404947 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa900018610 con 0x7fa90c1033d0 2024-09-19T06:15:55.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.828+0000 7fa91366c640 1 -- 172.21.15.53:0/3040404947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa90c1a3d10 con 0x7fa90c1033d0 2024-09-19T06:15:55.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.829+0000 7fa91366c640 1 -- 172.21.15.53:0/3040404947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa90c06a580 con 0x7fa90c1033d0 2024-09-19T06:15:55.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.829+0000 7fa8fe7fc640 1 -- 172.21.15.53:0/3040404947 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7fa900007b50 con 0x7fa90c1033d0 2024-09-19T06:15:55.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.830+0000 7fa8fe7fc640 1 --2- 172.21.15.53:0/3040404947 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fa8f806e2b0 0x7fa8f8070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:55.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.830+0000 7fa8fe7fc640 1 -- 172.21.15.53:0/3040404947 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5502+0+0 (secure 0 0 0) 0x7fa90008e350 con 0x7fa90c1033d0 2024-09-19T06:15:55.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.830+0000 7fa911408640 1 --2- 172.21.15.53:0/3040404947 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fa8f806e2b0 0x7fa8f8070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:55.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.831+0000 7fa911408640 1 --2- 172.21.15.53:0/3040404947 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fa8f806e2b0 0x7fa8f8070770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa908000c00 tx=0x7fa9080058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:55.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:55.835+0000 7fa8fe7fc640 1 -- 172.21.15.53:0/3040404947 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa900014030 con 0x7fa90c1033d0 2024-09-19T06:15:56.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:56.061+0000 7fa91366c640 1 -- 172.21.15.53:0/3040404947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fa90c105f80 con 0x7fa90c1033d0 2024-09-19T06:15:56.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:56.066+0000 7fa8fe7fc640 1 -- 172.21.15.53:0/3040404947 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) ==== 74+0+176565 (secure 0 0 0) 0x7fa90005bc00 con 0x7fa90c1033d0 2024-09-19T06:15:56.070 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:15:56.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:56.069+0000 7fa91366c640 1 -- 172.21.15.53:0/3040404947 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fa8f806e2b0 msgr2=0x7fa8f8070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:56.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:56.069+0000 7fa91366c640 1 --2- 172.21.15.53:0/3040404947 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fa8f806e2b0 0x7fa8f8070770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa908000c00 tx=0x7fa9080058d0 comp rx=0 tx=0).stop 2024-09-19T06:15:56.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:56.069+0000 7fa91366c640 1 -- 172.21.15.53:0/3040404947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa90c1033d0 msgr2=0x7fa90c100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:56.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:56.069+0000 7fa91366c640 1 --2- 172.21.15.53:0/3040404947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa90c1033d0 0x7fa90c100c50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa90000c8f0 tx=0x7fa90000cdb0 comp rx=0 tx=0).stop 2024-09-19T06:15:56.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:56.069+0000 7fa91366c640 1 -- 172.21.15.53:0/3040404947 shutdown_connections 2024-09-19T06:15:56.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:56.069+0000 7fa91366c640 1 --2- 172.21.15.53:0/3040404947 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fa8f806e2b0 0x7fa8f8070770 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:56.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:56.069+0000 7fa91366c640 1 --2- 172.21.15.53:0/3040404947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa90c1033d0 0x7fa90c100c50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:56.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:56.069+0000 7fa91366c640 1 --2- 172.21.15.53:0/3040404947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa90c102ab0 0x7fa90c100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:56.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:56.069+0000 7fa91366c640 1 -- 172.21.15.53:0/3040404947 >> 172.21.15.53:0/3040404947 conn(0x7fa90c0fc550 msgr2=0x7fa90c108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:56.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:56.069+0000 7fa91366c640 1 -- 172.21.15.53:0/3040404947 shutdown_connections 2024-09-19T06:15:56.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:56.069+0000 7fa91366c640 1 -- 172.21.15.53:0/3040404947 wait complete. 2024-09-19T06:15:56.646 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:56 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/3040404947' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T06:15:56.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:56 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/3040404947' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T06:15:56.750 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":19,"flags":0,"active_gid":14217,"active_name":"smithi053.trakvg","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3884355752},{"type":"v1","addr":"172.21.15.53:6801","nonce":3884355752}]},"active_addr":"172.21.15.53:6801/3884355752","active_change":"2024-09-19T06:11:28.721133+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":14244,"name":"smithi155.piymgv","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.53:8443/","prometheus":"http://172.21.15.53:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.53:0","nonce":1381499482}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.53:0","nonce":3335047243}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.53:0","nonce":877533156}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.53:0","nonce":2965073451}]}]} 2024-09-19T06:15:56.754 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-19T06:15:56.754 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-19T06:15:56.754 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:15:57.014 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:15:57.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.439+0000 7f7de33a2640 1 -- 172.21.15.53:0/1993055873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ddc101900 msgr2=0x7f7ddc101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:57.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.439+0000 7f7de33a2640 1 --2- 172.21.15.53:0/1993055873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ddc101900 0x7f7ddc101d60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc009880 tx=0x7f7dcc02a4a0 comp rx=0 tx=0).stop 2024-09-19T06:15:57.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.439+0000 7f7de33a2640 1 -- 172.21.15.53:0/1993055873 shutdown_connections 2024-09-19T06:15:57.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.439+0000 7f7de33a2640 1 --2- 172.21.15.53:0/1993055873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ddc101900 0x7f7ddc101d60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:57.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.439+0000 7f7de33a2640 1 --2- 172.21.15.53:0/1993055873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ddc107900 0x7f7ddc107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:57.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.439+0000 7f7de33a2640 1 -- 172.21.15.53:0/1993055873 >> 172.21.15.53:0/1993055873 conn(0x7f7ddc0fd600 msgr2=0x7f7ddc0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:57.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.439+0000 7f7de33a2640 1 -- 172.21.15.53:0/1993055873 shutdown_connections 2024-09-19T06:15:57.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.439+0000 7f7de33a2640 1 -- 172.21.15.53:0/1993055873 wait complete. 2024-09-19T06:15:57.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.440+0000 7f7de33a2640 1 Processor -- start 2024-09-19T06:15:57.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.440+0000 7f7de33a2640 1 -- start start 2024-09-19T06:15:57.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.440+0000 7f7de33a2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ddc101900 0x7f7ddc19d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:57.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.440+0000 7f7de33a2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ddc107900 0x7f7ddc19dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:57.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.440+0000 7f7de33a2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7ddc10eda0 con 0x7f7ddc101900 2024-09-19T06:15:57.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.440+0000 7f7de33a2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7ddc10ec20 con 0x7f7ddc107900 2024-09-19T06:15:57.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.440+0000 7f7de113e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ddc101900 0x7f7ddc19d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:57.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.440+0000 7f7de113e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ddc101900 0x7f7ddc19d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34032/0 (socket says 172.21.15.53:34032) 2024-09-19T06:15:57.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.440+0000 7f7de113e640 1 -- 172.21.15.53:0/3729724673 learned_addr learned my addr 172.21.15.53:0/3729724673 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:15:57.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.440+0000 7f7de093d640 1 --2- 172.21.15.53:0/3729724673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ddc107900 0x7f7ddc19dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:57.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.441+0000 7f7de113e640 1 -- 172.21.15.53:0/3729724673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ddc107900 msgr2=0x7f7ddc19dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:57.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.441+0000 7f7de113e640 1 --2- 172.21.15.53:0/3729724673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ddc107900 0x7f7ddc19dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:57.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.441+0000 7f7de113e640 1 -- 172.21.15.53:0/3729724673 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ddc197800 con 0x7f7ddc101900 2024-09-19T06:15:57.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.441+0000 7f7de113e640 1 --2- 172.21.15.53:0/3729724673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ddc101900 0x7f7ddc19d680 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7dd800b420 tx=0x7f7dd800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:57.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.441+0000 7f7dd27fc640 1 -- 172.21.15.53:0/3729724673 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7dd8011630 con 0x7f7ddc101900 2024-09-19T06:15:57.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.441+0000 7f7dd27fc640 1 -- 172.21.15.53:0/3729724673 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7dd8016070 con 0x7f7ddc101900 2024-09-19T06:15:57.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.441+0000 7f7de33a2640 1 -- 172.21.15.53:0/3729724673 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ddc197af0 con 0x7f7ddc101900 2024-09-19T06:15:57.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.441+0000 7f7dd27fc640 1 -- 172.21.15.53:0/3729724673 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7dd8010c90 con 0x7f7ddc101900 2024-09-19T06:15:57.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.441+0000 7f7de33a2640 1 -- 172.21.15.53:0/3729724673 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ddc198050 con 0x7f7ddc101900 2024-09-19T06:15:57.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.442+0000 7f7de33a2640 1 -- 172.21.15.53:0/3729724673 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dc0005290 con 0x7f7ddc101900 2024-09-19T06:15:57.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.443+0000 7f7dd27fc640 1 -- 172.21.15.53:0/3729724673 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f7dd8010380 con 0x7f7ddc101900 2024-09-19T06:15:57.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.443+0000 7f7dd27fc640 1 --2- 172.21.15.53:0/3729724673 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f7dc806e2b0 0x7f7dc8070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:57.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.444+0000 7f7de093d640 1 --2- 172.21.15.53:0/3729724673 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f7dc806e2b0 0x7f7dc8070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:57.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.444+0000 7f7dd27fc640 1 -- 172.21.15.53:0/3729724673 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5502+0+0 (secure 0 0 0) 0x7f7dd808d9d0 con 0x7f7ddc101900 2024-09-19T06:15:57.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.444+0000 7f7de093d640 1 --2- 172.21.15.53:0/3729724673 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f7dc806e2b0 0x7f7dc8070770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc198b50 tx=0x7f7dcc005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:57.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.448+0000 7f7dd27fc640 1 -- 172.21.15.53:0/3729724673 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7dd805b280 con 0x7f7ddc101900 2024-09-19T06:15:57.630 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.627+0000 7f7de33a2640 1 -- 172.21.15.53:0/3729724673 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7dc0005580 con 0x7f7ddc101900 2024-09-19T06:15:57.630 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.628+0000 7f7dd27fc640 1 -- 172.21.15.53:0/3729724673 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+13899 (secure 0 0 0) 0x7f7dd805ef30 con 0x7f7ddc101900 2024-09-19T06:15:57.631 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:15:57.631 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":60,"fsid":"9f2876d0-764d-11ef-bceb-c7b262605968","created":"2024-09-19T06:09:26.797853+0000","modified":"2024-09-19T06:15:51.817727+0000","last_up_change":"2024-09-19T06:15:47.809832+0000","last_in_change":"2024-09-19T06:15:26.420480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T06:13:39.241335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"35349d4a-0bde-46f6-8d5f-a019f0f89095","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1456825097},{"type":"v1","addr":"172.21.15.53:6803","nonce":1456825097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1456825097},{"type":"v1","addr":"172.21.15.53:6805","nonce":1456825097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1456825097},{"type":"v1","addr":"172.21.15.53:6809","nonce":1456825097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1456825097},{"type":"v1","addr":"172.21.15.53:6807","nonce":1456825097}]},"public_addr":"172.21.15.53:6803/1456825097","cluster_addr":"172.21.15.53:6805/1456825097","heartbeat_back_addr":"172.21.15.53:6809/1456825097","heartbeat_front_addr":"172.21.15.53:6807/1456825097","state":["exists","up"]},{"osd":1,"uuid":"832c2772-a460-4691-8db7-ad1d0c48cc9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3638097004},{"type":"v1","addr":"172.21.15.53:6811","nonce":3638097004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3638097004},{"type":"v1","addr":"172.21.15.53:6813","nonce":3638097004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3638097004},{"type":"v1","addr":"172.21.15.53:6817","nonce":3638097004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3638097004},{"type":"v1","addr":"172.21.15.53:6815","nonce":3638097004}]},"public_addr":"172.21.15.53:6811/3638097004","cluster_addr":"172.21.15.53:6813/3638097004","heartbeat_back_addr":"172.21.15.53:6817/3638097004","heartbeat_front_addr":"172.21.15.53:6815/3638097004","state":["exists","up"]},{"osd":2,"uuid":"57282117-559f-4495-b85f-90b68b2439ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":1988602192},{"type":"v1","addr":"172.21.15.53:6819","nonce":1988602192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":1988602192},{"type":"v1","addr":"172.21.15.53:6821","nonce":1988602192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1988602192},{"type":"v1","addr":"172.21.15.53:6825","nonce":1988602192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":1988602192},{"type":"v1","addr":"172.21.15.53:6823","nonce":1988602192}]},"public_addr":"172.21.15.53:6819/1988602192","cluster_addr":"172.21.15.53:6821/1988602192","heartbeat_back_addr":"172.21.15.53:6825/1988602192","heartbeat_front_addr":"172.21.15.53:6823/1988602192","state":["exists","up"]},{"osd":3,"uuid":"e6bf8cf0-2419-4c2a-b562-9221a117050c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2604012854},{"type":"v1","addr":"172.21.15.53:6827","nonce":2604012854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2604012854},{"type":"v1","addr":"172.21.15.53:6829","nonce":2604012854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2604012854},{"type":"v1","addr":"172.21.15.53:6833","nonce":2604012854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2604012854},{"type":"v1","addr":"172.21.15.53:6831","nonce":2604012854}]},"public_addr":"172.21.15.53:6827/2604012854","cluster_addr":"172.21.15.53:6829/2604012854","heartbeat_back_addr":"172.21.15.53:6833/2604012854","heartbeat_front_addr":"172.21.15.53:6831/2604012854","state":["exists","up"]},{"osd":4,"uuid":"7adedf0a-6fbc-42bc-8996-ba482b638929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2046029108},{"type":"v1","addr":"172.21.15.155:6801","nonce":2046029108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2046029108},{"type":"v1","addr":"172.21.15.155:6803","nonce":2046029108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2046029108},{"type":"v1","addr":"172.21.15.155:6807","nonce":2046029108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2046029108},{"type":"v1","addr":"172.21.15.155:6805","nonce":2046029108}]},"public_addr":"172.21.15.155:6801/2046029108","cluster_addr":"172.21.15.155:6803/2046029108","heartbeat_back_addr":"172.21.15.155:6807/2046029108","heartbeat_front_addr":"172.21.15.155:6805/2046029108","state":["exists","up"]},{"osd":5,"uuid":"c664e57c-20b5-46d4-8830-70780a91d6dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":464507356},{"type":"v1","addr":"172.21.15.155:6809","nonce":464507356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":464507356},{"type":"v1","addr":"172.21.15.155:6811","nonce":464507356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":464507356},{"type":"v1","addr":"172.21.15.155:6815","nonce":464507356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":464507356},{"type":"v1","addr":"172.21.15.155:6813","nonce":464507356}]},"public_addr":"172.21.15.155:6809/464507356","cluster_addr":"172.21.15.155:6811/464507356","heartbeat_back_addr":"172.21.15.155:6815/464507356","heartbeat_front_addr":"172.21.15.155:6813/464507356","state":["exists","up"]},{"osd":6,"uuid":"bc8a88f5-75ab-4482-88e0-5cc5e6f06db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2574655677},{"type":"v1","addr":"172.21.15.155:6817","nonce":2574655677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2574655677},{"type":"v1","addr":"172.21.15.155:6819","nonce":2574655677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2574655677},{"type":"v1","addr":"172.21.15.155:6823","nonce":2574655677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2574655677},{"type":"v1","addr":"172.21.15.155:6821","nonce":2574655677}]},"public_addr":"172.21.15.155:6817/2574655677","cluster_addr":"172.21.15.155:6819/2574655677","heartbeat_back_addr":"172.21.15.155:6823/2574655677","heartbeat_front_addr":"172.21.15.155:6821/2574655677","state":["exists","up"]},{"osd":7,"uuid":"b91759fe-f27c-4ab2-9bd3-b5961b358d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2269161534},{"type":"v1","addr":"172.21.15.155:6825","nonce":2269161534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2269161534},{"type":"v1","addr":"172.21.15.155:6827","nonce":2269161534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2269161534},{"type":"v1","addr":"172.21.15.155:6831","nonce":2269161534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2269161534},{"type":"v1","addr":"172.21.15.155:6829","nonce":2269161534}]},"public_addr":"172.21.15.155:6825/2269161534","cluster_addr":"172.21.15.155:6827/2269161534","heartbeat_back_addr":"172.21.15.155:6831/2269161534","heartbeat_front_addr":"172.21.15.155:6829/2269161534","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:12:43.760347+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:13:10.493898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:13:37.244856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:14:05.006717+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:14:29.801028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:14:55.095341+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:15:20.683202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:15:45.635704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6801/3684090301":"2024-09-20T06:11:28.721009+0000","172.21.15.53:6800/475342876":"2024-09-20T06:09:58.271775+0000","172.21.15.53:6801/475342876":"2024-09-20T06:09:58.271775+0000","172.21.15.53:0/3753705881":"2024-09-20T06:11:28.721009+0000","172.21.15.53:0/4216592427":"2024-09-20T06:09:58.271775+0000","172.21.15.53:0/1618433917":"2024-09-20T06:10:30.209033+0000","172.21.15.53:0/623598636":"2024-09-20T06:09:58.271775+0000","172.21.15.53:6800/3684090301":"2024-09-20T06:11:28.721009+0000","172.21.15.53:6800/681468213":"2024-09-20T06:10:30.209033+0000","172.21.15.53:0/597749918":"2024-09-20T06:10:30.209033+0000","172.21.15.53:0/21853845":"2024-09-20T06:11:28.721009+0000","172.21.15.53:0/3515303955":"2024-09-20T06:09:58.271775+0000","172.21.15.53:6801/681468213":"2024-09-20T06:10:30.209033+0000","172.21.15.53:0/1729980812":"2024-09-20T06:11:28.721009+0000","172.21.15.53:0/372276981":"2024-09-20T06:10:30.209033+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:15:57.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.630+0000 7f7de33a2640 1 -- 172.21.15.53:0/3729724673 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f7dc806e2b0 msgr2=0x7f7dc8070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:57.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.630+0000 7f7de33a2640 1 --2- 172.21.15.53:0/3729724673 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f7dc806e2b0 0x7f7dc8070770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc198b50 tx=0x7f7dcc005a20 comp rx=0 tx=0).stop 2024-09-19T06:15:57.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.630+0000 7f7de33a2640 1 -- 172.21.15.53:0/3729724673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ddc101900 msgr2=0x7f7ddc19d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:57.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.630+0000 7f7de33a2640 1 --2- 172.21.15.53:0/3729724673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ddc101900 0x7f7ddc19d680 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7dd800b420 tx=0x7f7dd800b8e0 comp rx=0 tx=0).stop 2024-09-19T06:15:57.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.630+0000 7f7de33a2640 1 -- 172.21.15.53:0/3729724673 shutdown_connections 2024-09-19T06:15:57.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.630+0000 7f7de33a2640 1 --2- 172.21.15.53:0/3729724673 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f7dc806e2b0 0x7f7dc8070770 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:57.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.630+0000 7f7de33a2640 1 --2- 172.21.15.53:0/3729724673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ddc107900 0x7f7ddc19dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:57.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.630+0000 7f7de33a2640 1 --2- 172.21.15.53:0/3729724673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ddc101900 0x7f7ddc19d680 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:57.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.630+0000 7f7de33a2640 1 -- 172.21.15.53:0/3729724673 >> 172.21.15.53:0/3729724673 conn(0x7f7ddc0fd600 msgr2=0x7f7ddc0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:57.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.630+0000 7f7de33a2640 1 -- 172.21.15.53:0/3729724673 shutdown_connections 2024-09-19T06:15:57.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:57.630+0000 7f7de33a2640 1 -- 172.21.15.53:0/3729724673 wait complete. 2024-09-19T06:15:57.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:57 smithi155 ceph-mon[35292]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-19T06:15:57.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:57 smithi053 ceph-mon[27968]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-19T06:15:58.328 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-19T06:15:58.328 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:15:58.591 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:15:58.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:58 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/3729724673' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:15:58.717 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:58 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/3729724673' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:15:59.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.022+0000 7f241a519640 1 -- 172.21.15.53:0/278114087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2414107900 msgr2=0x7f2414107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:59.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.022+0000 7f241a519640 1 --2- 172.21.15.53:0/278114087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2414107900 0x7f2414107ce0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2404009880 tx=0x7f240402a4a0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.022+0000 7f241a519640 1 -- 172.21.15.53:0/278114087 shutdown_connections 2024-09-19T06:15:59.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.022+0000 7f241a519640 1 --2- 172.21.15.53:0/278114087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2414101900 0x7f2414101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.022+0000 7f241a519640 1 --2- 172.21.15.53:0/278114087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2414107900 0x7f2414107ce0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.022+0000 7f241a519640 1 -- 172.21.15.53:0/278114087 >> 172.21.15.53:0/278114087 conn(0x7f24140fd600 msgr2=0x7f24140ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:59.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.022+0000 7f241a519640 1 -- 172.21.15.53:0/278114087 shutdown_connections 2024-09-19T06:15:59.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.022+0000 7f241a519640 1 -- 172.21.15.53:0/278114087 wait complete. 2024-09-19T06:15:59.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.023+0000 7f241a519640 1 Processor -- start 2024-09-19T06:15:59.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.023+0000 7f241a519640 1 -- start start 2024-09-19T06:15:59.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.023+0000 7f241a519640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2414101900 0x7f24140744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:59.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.023+0000 7f241a519640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2414107900 0x7f2414074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:59.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.023+0000 7f241a519640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f241410eda0 con 0x7f2414101900 2024-09-19T06:15:59.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.023+0000 7f241a519640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f241410ec20 con 0x7f2414107900 2024-09-19T06:15:59.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.023+0000 7f2413fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2414101900 0x7f24140744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:59.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.023+0000 7f2413fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2414101900 0x7f24140744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34040/0 (socket says 172.21.15.53:34040) 2024-09-19T06:15:59.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.023+0000 7f2413fff640 1 -- 172.21.15.53:0/3251643726 learned_addr learned my addr 172.21.15.53:0/3251643726 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:15:59.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.023+0000 7f24137fe640 1 --2- 172.21.15.53:0/3251643726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2414107900 0x7f2414074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:59.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.024+0000 7f2413fff640 1 -- 172.21.15.53:0/3251643726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2414107900 msgr2=0x7f2414074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:59.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.024+0000 7f2413fff640 1 --2- 172.21.15.53:0/3251643726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2414107900 0x7f2414074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.024+0000 7f2413fff640 1 -- 172.21.15.53:0/3251643726 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2414074f70 con 0x7f2414101900 2024-09-19T06:15:59.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.024+0000 7f24137fe640 1 --2- 172.21.15.53:0/3251643726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2414107900 0x7f2414074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:15:59.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.024+0000 7f2413fff640 1 --2- 172.21.15.53:0/3251643726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2414101900 0x7f24140744f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2404006010 tx=0x7f24040048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:59.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.024+0000 7f24117fa640 1 -- 172.21.15.53:0/3251643726 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f240403f070 con 0x7f2414101900 2024-09-19T06:15:59.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.024+0000 7f24117fa640 1 -- 172.21.15.53:0/3251643726 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2404003aa0 con 0x7f2414101900 2024-09-19T06:15:59.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.024+0000 7f24117fa640 1 -- 172.21.15.53:0/3251643726 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2404035030 con 0x7f2414101900 2024-09-19T06:15:59.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.024+0000 7f241a519640 1 -- 172.21.15.53:0/3251643726 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2414075200 con 0x7f2414101900 2024-09-19T06:15:59.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.024+0000 7f241a519640 1 -- 172.21.15.53:0/3251643726 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24141a3d70 con 0x7f2414101900 2024-09-19T06:15:59.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.025+0000 7f241a519640 1 -- 172.21.15.53:0/3251643726 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2414103040 con 0x7f2414101900 2024-09-19T06:15:59.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.025+0000 7f24117fa640 1 -- 172.21.15.53:0/3251643726 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f2404042070 con 0x7f2414101900 2024-09-19T06:15:59.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.026+0000 7f24117fa640 1 --2- 172.21.15.53:0/3251643726 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f23fc06e2b0 0x7f23fc070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:59.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.026+0000 7f24117fa640 1 -- 172.21.15.53:0/3251643726 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5502+0+0 (secure 0 0 0) 0x7f24040aedb0 con 0x7f2414101900 2024-09-19T06:15:59.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.026+0000 7f24137fe640 1 --2- 172.21.15.53:0/3251643726 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f23fc06e2b0 0x7f23fc070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:59.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.027+0000 7f24137fe640 1 --2- 172.21.15.53:0/3251643726 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f23fc06e2b0 0x7f23fc070770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2408006fd0 tx=0x7f2408008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:59.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.031+0000 7f24117fa640 1 -- 172.21.15.53:0/3251643726 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f240407c660 con 0x7f2414101900 2024-09-19T06:15:59.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.206+0000 7f241a519640 1 -- 172.21.15.53:0/3251643726 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2414109e70 con 0x7f2414101900 2024-09-19T06:15:59.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.207+0000 7f24117fa640 1 -- 172.21.15.53:0/3251643726 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+13899 (secure 0 0 0) 0x7f2404080310 con 0x7f2414101900 2024-09-19T06:15:59.209 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:15:59.209 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":60,"fsid":"9f2876d0-764d-11ef-bceb-c7b262605968","created":"2024-09-19T06:09:26.797853+0000","modified":"2024-09-19T06:15:51.817727+0000","last_up_change":"2024-09-19T06:15:47.809832+0000","last_in_change":"2024-09-19T06:15:26.420480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T06:13:39.241335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"35349d4a-0bde-46f6-8d5f-a019f0f89095","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1456825097},{"type":"v1","addr":"172.21.15.53:6803","nonce":1456825097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1456825097},{"type":"v1","addr":"172.21.15.53:6805","nonce":1456825097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1456825097},{"type":"v1","addr":"172.21.15.53:6809","nonce":1456825097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1456825097},{"type":"v1","addr":"172.21.15.53:6807","nonce":1456825097}]},"public_addr":"172.21.15.53:6803/1456825097","cluster_addr":"172.21.15.53:6805/1456825097","heartbeat_back_addr":"172.21.15.53:6809/1456825097","heartbeat_front_addr":"172.21.15.53:6807/1456825097","state":["exists","up"]},{"osd":1,"uuid":"832c2772-a460-4691-8db7-ad1d0c48cc9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3638097004},{"type":"v1","addr":"172.21.15.53:6811","nonce":3638097004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3638097004},{"type":"v1","addr":"172.21.15.53:6813","nonce":3638097004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3638097004},{"type":"v1","addr":"172.21.15.53:6817","nonce":3638097004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3638097004},{"type":"v1","addr":"172.21.15.53:6815","nonce":3638097004}]},"public_addr":"172.21.15.53:6811/3638097004","cluster_addr":"172.21.15.53:6813/3638097004","heartbeat_back_addr":"172.21.15.53:6817/3638097004","heartbeat_front_addr":"172.21.15.53:6815/3638097004","state":["exists","up"]},{"osd":2,"uuid":"57282117-559f-4495-b85f-90b68b2439ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":1988602192},{"type":"v1","addr":"172.21.15.53:6819","nonce":1988602192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":1988602192},{"type":"v1","addr":"172.21.15.53:6821","nonce":1988602192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1988602192},{"type":"v1","addr":"172.21.15.53:6825","nonce":1988602192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":1988602192},{"type":"v1","addr":"172.21.15.53:6823","nonce":1988602192}]},"public_addr":"172.21.15.53:6819/1988602192","cluster_addr":"172.21.15.53:6821/1988602192","heartbeat_back_addr":"172.21.15.53:6825/1988602192","heartbeat_front_addr":"172.21.15.53:6823/1988602192","state":["exists","up"]},{"osd":3,"uuid":"e6bf8cf0-2419-4c2a-b562-9221a117050c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2604012854},{"type":"v1","addr":"172.21.15.53:6827","nonce":2604012854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2604012854},{"type":"v1","addr":"172.21.15.53:6829","nonce":2604012854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2604012854},{"type":"v1","addr":"172.21.15.53:6833","nonce":2604012854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2604012854},{"type":"v1","addr":"172.21.15.53:6831","nonce":2604012854}]},"public_addr":"172.21.15.53:6827/2604012854","cluster_addr":"172.21.15.53:6829/2604012854","heartbeat_back_addr":"172.21.15.53:6833/2604012854","heartbeat_front_addr":"172.21.15.53:6831/2604012854","state":["exists","up"]},{"osd":4,"uuid":"7adedf0a-6fbc-42bc-8996-ba482b638929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2046029108},{"type":"v1","addr":"172.21.15.155:6801","nonce":2046029108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2046029108},{"type":"v1","addr":"172.21.15.155:6803","nonce":2046029108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2046029108},{"type":"v1","addr":"172.21.15.155:6807","nonce":2046029108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2046029108},{"type":"v1","addr":"172.21.15.155:6805","nonce":2046029108}]},"public_addr":"172.21.15.155:6801/2046029108","cluster_addr":"172.21.15.155:6803/2046029108","heartbeat_back_addr":"172.21.15.155:6807/2046029108","heartbeat_front_addr":"172.21.15.155:6805/2046029108","state":["exists","up"]},{"osd":5,"uuid":"c664e57c-20b5-46d4-8830-70780a91d6dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":464507356},{"type":"v1","addr":"172.21.15.155:6809","nonce":464507356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":464507356},{"type":"v1","addr":"172.21.15.155:6811","nonce":464507356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":464507356},{"type":"v1","addr":"172.21.15.155:6815","nonce":464507356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":464507356},{"type":"v1","addr":"172.21.15.155:6813","nonce":464507356}]},"public_addr":"172.21.15.155:6809/464507356","cluster_addr":"172.21.15.155:6811/464507356","heartbeat_back_addr":"172.21.15.155:6815/464507356","heartbeat_front_addr":"172.21.15.155:6813/464507356","state":["exists","up"]},{"osd":6,"uuid":"bc8a88f5-75ab-4482-88e0-5cc5e6f06db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2574655677},{"type":"v1","addr":"172.21.15.155:6817","nonce":2574655677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2574655677},{"type":"v1","addr":"172.21.15.155:6819","nonce":2574655677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2574655677},{"type":"v1","addr":"172.21.15.155:6823","nonce":2574655677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2574655677},{"type":"v1","addr":"172.21.15.155:6821","nonce":2574655677}]},"public_addr":"172.21.15.155:6817/2574655677","cluster_addr":"172.21.15.155:6819/2574655677","heartbeat_back_addr":"172.21.15.155:6823/2574655677","heartbeat_front_addr":"172.21.15.155:6821/2574655677","state":["exists","up"]},{"osd":7,"uuid":"b91759fe-f27c-4ab2-9bd3-b5961b358d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2269161534},{"type":"v1","addr":"172.21.15.155:6825","nonce":2269161534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2269161534},{"type":"v1","addr":"172.21.15.155:6827","nonce":2269161534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2269161534},{"type":"v1","addr":"172.21.15.155:6831","nonce":2269161534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2269161534},{"type":"v1","addr":"172.21.15.155:6829","nonce":2269161534}]},"public_addr":"172.21.15.155:6825/2269161534","cluster_addr":"172.21.15.155:6827/2269161534","heartbeat_back_addr":"172.21.15.155:6831/2269161534","heartbeat_front_addr":"172.21.15.155:6829/2269161534","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:12:43.760347+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:13:10.493898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:13:37.244856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:14:05.006717+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:14:29.801028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:14:55.095341+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:15:20.683202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:15:45.635704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6801/3684090301":"2024-09-20T06:11:28.721009+0000","172.21.15.53:6800/475342876":"2024-09-20T06:09:58.271775+0000","172.21.15.53:6801/475342876":"2024-09-20T06:09:58.271775+0000","172.21.15.53:0/3753705881":"2024-09-20T06:11:28.721009+0000","172.21.15.53:0/4216592427":"2024-09-20T06:09:58.271775+0000","172.21.15.53:0/1618433917":"2024-09-20T06:10:30.209033+0000","172.21.15.53:0/623598636":"2024-09-20T06:09:58.271775+0000","172.21.15.53:6800/3684090301":"2024-09-20T06:11:28.721009+0000","172.21.15.53:6800/681468213":"2024-09-20T06:10:30.209033+0000","172.21.15.53:0/597749918":"2024-09-20T06:10:30.209033+0000","172.21.15.53:0/21853845":"2024-09-20T06:11:28.721009+0000","172.21.15.53:0/3515303955":"2024-09-20T06:09:58.271775+0000","172.21.15.53:6801/681468213":"2024-09-20T06:10:30.209033+0000","172.21.15.53:0/1729980812":"2024-09-20T06:11:28.721009+0000","172.21.15.53:0/372276981":"2024-09-20T06:10:30.209033+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:15:59.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.210+0000 7f241a519640 1 -- 172.21.15.53:0/3251643726 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f23fc06e2b0 msgr2=0x7f23fc070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:59.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.210+0000 7f241a519640 1 --2- 172.21.15.53:0/3251643726 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f23fc06e2b0 0x7f23fc070770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2408006fd0 tx=0x7f2408008040 comp rx=0 tx=0).stop 2024-09-19T06:15:59.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.210+0000 7f241a519640 1 -- 172.21.15.53:0/3251643726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2414101900 msgr2=0x7f24140744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:59.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.210+0000 7f241a519640 1 --2- 172.21.15.53:0/3251643726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2414101900 0x7f24140744f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2404006010 tx=0x7f24040048f0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.210+0000 7f241a519640 1 -- 172.21.15.53:0/3251643726 shutdown_connections 2024-09-19T06:15:59.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.210+0000 7f241a519640 1 --2- 172.21.15.53:0/3251643726 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f23fc06e2b0 0x7f23fc070770 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.210+0000 7f241a519640 1 --2- 172.21.15.53:0/3251643726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2414107900 0x7f2414074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.210+0000 7f241a519640 1 --2- 172.21.15.53:0/3251643726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2414101900 0x7f24140744f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.210+0000 7f241a519640 1 -- 172.21.15.53:0/3251643726 >> 172.21.15.53:0/3251643726 conn(0x7f24140fd600 msgr2=0x7f24140fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:59.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.210+0000 7f241a519640 1 -- 172.21.15.53:0/3251643726 shutdown_connections 2024-09-19T06:15:59.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:15:59.210+0000 7f241a519640 1 -- 172.21.15.53:0/3251643726 wait complete. 2024-09-19T06:15:59.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:15:59.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:59 smithi155 ceph-mon[35292]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T06:15:59.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:15:59 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/3251643726' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:15:59.820 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:15:59.820 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:59 smithi053 ceph-mon[27968]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T06:15:59.820 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:15:59 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/3251643726' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:15:59.850 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-19T06:15:59.851 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-19T06:15:59.851 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-19T06:15:59.851 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-19T06:15:59.852 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-19T06:15:59.852 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-19T06:15:59.853 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-19T06:15:59.853 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-19T06:16:00.428 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:00.430 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:00.431 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:00.432 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:00.433 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:00.436 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:00.436 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:00.440 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:01.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:00 smithi053 ceph-mon[27968]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T06:16:01.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:00 smithi155 ceph-mon[35292]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T06:16:01.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.875+0000 7f41d64c0640 1 -- 172.21.15.53:0/2131059291 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d0107900 msgr2=0x7f41d0107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:01.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.875+0000 7f41d64c0640 1 --2- 172.21.15.53:0/2131059291 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d0107900 0x7f41d0107ce0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009880 tx=0x7f41c002a4a0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.875+0000 7f41d64c0640 1 -- 172.21.15.53:0/2131059291 shutdown_connections 2024-09-19T06:16:01.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.875+0000 7f41d64c0640 1 --2- 172.21.15.53:0/2131059291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41d0101900 0x7f41d0101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.875+0000 7f41d64c0640 1 --2- 172.21.15.53:0/2131059291 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d0107900 0x7f41d0107ce0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.875+0000 7f41d64c0640 1 -- 172.21.15.53:0/2131059291 >> 172.21.15.53:0/2131059291 conn(0x7f41d00fd5e0 msgr2=0x7f41d00ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:01.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.875+0000 7f41d64c0640 1 -- 172.21.15.53:0/2131059291 shutdown_connections 2024-09-19T06:16:01.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.875+0000 7f41d64c0640 1 -- 172.21.15.53:0/2131059291 wait complete. 2024-09-19T06:16:01.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.876+0000 7f41d64c0640 1 Processor -- start 2024-09-19T06:16:01.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.876+0000 7f41d64c0640 1 -- start start 2024-09-19T06:16:01.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.876+0000 7f41d64c0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41d0101900 0x7f41d019f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.876+0000 7f41d64c0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d0107900 0x7f41d019fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.876+0000 7f41d64c0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f41d010eda0 con 0x7f41d0107900 2024-09-19T06:16:01.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.876+0000 7f41d64c0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f41d010ec20 con 0x7f41d0101900 2024-09-19T06:16:01.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.876+0000 7f41cf7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d0107900 0x7f41d019fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.876+0000 7f41cf7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d0107900 0x7f41d019fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41350/0 (socket says 172.21.15.53:41350) 2024-09-19T06:16:01.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.877+0000 7f41cf7fe640 1 -- 172.21.15.53:0/1415496972 learned_addr learned my addr 172.21.15.53:0/1415496972 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:01.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.877+0000 7f41cffff640 1 --2- 172.21.15.53:0/1415496972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41d0101900 0x7f41d019f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.877+0000 7f41cf7fe640 1 -- 172.21.15.53:0/1415496972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41d0101900 msgr2=0x7f41d019f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:01.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.877+0000 7f41cf7fe640 1 --2- 172.21.15.53:0/1415496972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41d0101900 0x7f41d019f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.877+0000 7f41cf7fe640 1 -- 172.21.15.53:0/1415496972 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d01a0420 con 0x7f41d0107900 2024-09-19T06:16:01.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.877+0000 7f41cf7fe640 1 --2- 172.21.15.53:0/1415496972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d0107900 0x7f41d019fd70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f41c400b420 tx=0x7f41c400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:01.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.877+0000 7f41cd7fa640 1 -- 172.21.15.53:0/1415496972 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f41c4011630 con 0x7f41d0107900 2024-09-19T06:16:01.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.877+0000 7f41cd7fa640 1 -- 172.21.15.53:0/1415496972 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41c4016070 con 0x7f41d0107900 2024-09-19T06:16:01.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.877+0000 7f41d64c0640 1 -- 172.21.15.53:0/1415496972 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d0199b10 con 0x7f41d0107900 2024-09-19T06:16:01.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.877+0000 7f41cd7fa640 1 -- 172.21.15.53:0/1415496972 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f41c4010c90 con 0x7f41d0107900 2024-09-19T06:16:01.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.877+0000 7f41d64c0640 1 -- 172.21.15.53:0/1415496972 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d019a070 con 0x7f41d0107900 2024-09-19T06:16:01.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.878+0000 7f41d64c0640 1 -- 172.21.15.53:0/1415496972 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f41d019a690 con 0x7f41d0107900 2024-09-19T06:16:01.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.879+0000 7f41cd7fa640 1 -- 172.21.15.53:0/1415496972 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f41c40117d0 con 0x7f41d0107900 2024-09-19T06:16:01.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.879+0000 7f41cd7fa640 1 --2- 172.21.15.53:0/1415496972 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f41b806e310 0x7f41b80707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.879+0000 7f41cd7fa640 1 -- 172.21.15.53:0/1415496972 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5502+0+0 (secure 0 0 0) 0x7f41c408da10 con 0x7f41d0107900 2024-09-19T06:16:01.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.879+0000 7f41cffff640 1 --2- 172.21.15.53:0/1415496972 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f41b806e310 0x7f41b80707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.880+0000 7f41cd7fa640 1 --2- 172.21.15.53:0/1415496972 >> [v2:172.21.15.53:6818/1988602192,v1:172.21.15.53:6819/1988602192] conn(0x7f41b80746d0 0x7f41b8076af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.880+0000 7f41cd7fa640 1 -- 172.21.15.53:0/1415496972 --> [v2:172.21.15.53:6818/1988602192,v1:172.21.15.53:6819/1988602192] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f41c4014e50 con 0x7f41b80746d0 2024-09-19T06:16:01.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.880+0000 7f41cd7fa640 1 -- 172.21.15.53:0/1415496972 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f41c40922a0 con 0x7f41d0107900 2024-09-19T06:16:01.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.880+0000 7f41d4a5d640 1 --2- 172.21.15.53:0/1415496972 >> [v2:172.21.15.53:6818/1988602192,v1:172.21.15.53:6819/1988602192] conn(0x7f41b80746d0 0x7f41b8076af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.880+0000 7f41cffff640 1 --2- 172.21.15.53:0/1415496972 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f41b806e310 0x7f41b80707d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f41c0000c00 tx=0x7f41c0005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:01.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.880+0000 7f41d4a5d640 1 --2- 172.21.15.53:0/1415496972 >> [v2:172.21.15.53:6818/1988602192,v1:172.21.15.53:6819/1988602192] conn(0x7f41b80746d0 0x7f41b8076af0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:01.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.881+0000 7f41cd7fa640 1 -- 172.21.15.53:0/1415496972 <== osd.2 v2:172.21.15.53:6818/1988602192 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f41c4014e50 con 0x7f41b80746d0 2024-09-19T06:16:01.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.894+0000 7f41d64c0640 1 -- 172.21.15.53:0/1415496972 --> [v2:172.21.15.53:6818/1988602192,v1:172.21.15.53:6819/1988602192] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f41d0074ec0 con 0x7f41b80746d0 2024-09-19T06:16:01.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.895+0000 7f41cd7fa640 1 -- 172.21.15.53:0/1415496972 <== osd.2 v2:172.21.15.53:6818/1988602192 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f41d0074ec0 con 0x7f41b80746d0 2024-09-19T06:16:01.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.895+0000 7f41d64c0640 1 -- 172.21.15.53:0/1415496972 >> [v2:172.21.15.53:6818/1988602192,v1:172.21.15.53:6819/1988602192] conn(0x7f41b80746d0 msgr2=0x7f41b8076af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:01.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.895+0000 7f41d64c0640 1 --2- 172.21.15.53:0/1415496972 >> [v2:172.21.15.53:6818/1988602192,v1:172.21.15.53:6819/1988602192] conn(0x7f41b80746d0 0x7f41b8076af0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.895+0000 7f41d64c0640 1 -- 172.21.15.53:0/1415496972 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f41b806e310 msgr2=0x7f41b80707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:01.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.895+0000 7f41d64c0640 1 --2- 172.21.15.53:0/1415496972 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f41b806e310 0x7f41b80707d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f41c0000c00 tx=0x7f41c0005a20 comp rx=0 tx=0).stop 2024-09-19T06:16:01.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.895+0000 7f41d64c0640 1 -- 172.21.15.53:0/1415496972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d0107900 msgr2=0x7f41d019fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:01.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.895+0000 7f41d64c0640 1 --2- 172.21.15.53:0/1415496972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d0107900 0x7f41d019fd70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f41c400b420 tx=0x7f41c400b8e0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.895+0000 7f41d64c0640 1 -- 172.21.15.53:0/1415496972 shutdown_connections 2024-09-19T06:16:01.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.895+0000 7f41d64c0640 1 --2- 172.21.15.53:0/1415496972 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f41b806e310 0x7f41b80707d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.895+0000 7f41d64c0640 1 --2- 172.21.15.53:0/1415496972 >> [v2:172.21.15.53:6818/1988602192,v1:172.21.15.53:6819/1988602192] conn(0x7f41b80746d0 0x7f41b8076af0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.895+0000 7f41d64c0640 1 --2- 172.21.15.53:0/1415496972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d0107900 0x7f41d019fd70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.895+0000 7f41d64c0640 1 --2- 172.21.15.53:0/1415496972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41d0101900 0x7f41d019f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.895+0000 7f41d64c0640 1 -- 172.21.15.53:0/1415496972 >> 172.21.15.53:0/1415496972 conn(0x7f41d00fd5e0 msgr2=0x7f41d00744e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:01.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.895+0000 7f41d64c0640 1 -- 172.21.15.53:0/1415496972 shutdown_connections 2024-09-19T06:16:01.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:01.895+0000 7f41d64c0640 1 -- 172.21.15.53:0/1415496972 wait complete. 2024-09-19T06:16:02.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.429+0000 7f98d0d2e640 1 -- 172.21.15.53:0/1550666890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98cc101900 msgr2=0x7f98cc101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:02.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.429+0000 7f98d0d2e640 1 --2- 172.21.15.53:0/1550666890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98cc101900 0x7f98cc101d60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009880 tx=0x7f98c002a4a0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.429+0000 7f98d0d2e640 1 -- 172.21.15.53:0/1550666890 shutdown_connections 2024-09-19T06:16:02.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.429+0000 7f98d0d2e640 1 --2- 172.21.15.53:0/1550666890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98cc101900 0x7f98cc101d60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.429+0000 7f98d0d2e640 1 --2- 172.21.15.53:0/1550666890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98cc107900 0x7f98cc107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.429+0000 7f98d0d2e640 1 -- 172.21.15.53:0/1550666890 >> 172.21.15.53:0/1550666890 conn(0x7f98cc0fd600 msgr2=0x7f98cc0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:02.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.429+0000 7f98d0d2e640 1 -- 172.21.15.53:0/1550666890 shutdown_connections 2024-09-19T06:16:02.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.429+0000 7f98d0d2e640 1 -- 172.21.15.53:0/1550666890 wait complete. 2024-09-19T06:16:02.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.430+0000 7f98d0d2e640 1 Processor -- start 2024-09-19T06:16:02.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.430+0000 7f98d0d2e640 1 -- start start 2024-09-19T06:16:02.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.430+0000 7f98d0d2e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98cc101900 0x7f98cc19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:02.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.430+0000 7f98d0d2e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98cc107900 0x7f98cc19dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:02.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.430+0000 7f98d0d2e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc10eda0 con 0x7f98cc101900 2024-09-19T06:16:02.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.430+0000 7f98d0d2e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc10ec20 con 0x7f98cc107900 2024-09-19T06:16:02.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.430+0000 7f98ca59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98cc101900 0x7f98cc19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:02.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.430+0000 7f98ca59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98cc101900 0x7f98cc19d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41370/0 (socket says 172.21.15.53:41370) 2024-09-19T06:16:02.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.430+0000 7f98ca59c640 1 -- 172.21.15.53:0/3273644494 learned_addr learned my addr 172.21.15.53:0/3273644494 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:02.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.430+0000 7f98c9d9b640 1 --2- 172.21.15.53:0/3273644494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98cc107900 0x7f98cc19dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.430+0000 7f98ca59c640 1 -- 172.21.15.53:0/3273644494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98cc107900 msgr2=0x7f98cc19dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.430+0000 7f98ca59c640 1 --2- 172.21.15.53:0/3273644494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98cc107900 0x7f98cc19dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.430+0000 7f98ca59c640 1 -- 172.21.15.53:0/3273644494 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98cc19e290 con 0x7f98cc101900 2024-09-19T06:16:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.430+0000 7f98c9d9b640 1 --2- 172.21.15.53:0/3273644494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98cc107900 0x7f98cc19dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:16:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.431+0000 7f98ca59c640 1 --2- 172.21.15.53:0/3273644494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98cc101900 0x7f98cc19d6a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00b420 tx=0x7f98bc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.431+0000 7f98bb7fe640 1 -- 172.21.15.53:0/3273644494 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f98bc011630 con 0x7f98cc101900 2024-09-19T06:16:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.431+0000 7f98bb7fe640 1 -- 172.21.15.53:0/3273644494 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98bc016070 con 0x7f98cc101900 2024-09-19T06:16:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.431+0000 7f98d0d2e640 1 -- 172.21.15.53:0/3273644494 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98cc197980 con 0x7f98cc101900 2024-09-19T06:16:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.431+0000 7f98bb7fe640 1 -- 172.21.15.53:0/3273644494 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f98bc010c90 con 0x7f98cc101900 2024-09-19T06:16:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.431+0000 7f98d0d2e640 1 -- 172.21.15.53:0/3273644494 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98cc197ee0 con 0x7f98cc101900 2024-09-19T06:16:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.432+0000 7f98d0d2e640 1 -- 172.21.15.53:0/3273644494 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f98cc1984d0 con 0x7f98cc101900 2024-09-19T06:16:02.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.432+0000 7f98bb7fe640 1 -- 172.21.15.53:0/3273644494 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f98bc0117d0 con 0x7f98cc101900 2024-09-19T06:16:02.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.433+0000 7f98bb7fe640 1 --2- 172.21.15.53:0/3273644494 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f98b406e2b0 0x7f98b4070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:02.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.433+0000 7f98bb7fe640 1 -- 172.21.15.53:0/3273644494 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5502+0+0 (secure 0 0 0) 0x7f98bc08dab0 con 0x7f98cc101900 2024-09-19T06:16:02.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.433+0000 7f98c9d9b640 1 --2- 172.21.15.53:0/3273644494 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f98b406e2b0 0x7f98b4070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:02.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.433+0000 7f98bb7fe640 1 --2- 172.21.15.53:0/3273644494 >> [v2:172.21.15.155:6800/2046029108,v1:172.21.15.155:6801/2046029108] conn(0x7f98b4074670 0x7f98b4076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:02.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.433+0000 7f98bb7fe640 1 -- 172.21.15.53:0/3273644494 --> [v2:172.21.15.155:6800/2046029108,v1:172.21.15.155:6801/2046029108] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f98b4077140 con 0x7f98b4074670 2024-09-19T06:16:02.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.433+0000 7f98bb7fe640 1 -- 172.21.15.53:0/3273644494 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f98bc0922a0 con 0x7f98cc101900 2024-09-19T06:16:02.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.433+0000 7f98cad9d640 1 --2- 172.21.15.53:0/3273644494 >> [v2:172.21.15.155:6800/2046029108,v1:172.21.15.155:6801/2046029108] conn(0x7f98b4074670 0x7f98b4076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:02.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.433+0000 7f98c9d9b640 1 --2- 172.21.15.53:0/3273644494 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f98b406e2b0 0x7f98b4070770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f98cc199050 tx=0x7f98c0005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:02.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.434+0000 7f98cad9d640 1 --2- 172.21.15.53:0/3273644494 >> [v2:172.21.15.155:6800/2046029108,v1:172.21.15.155:6801/2046029108] conn(0x7f98b4074670 0x7f98b4076a90 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:02.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.435+0000 7f98bb7fe640 1 -- 172.21.15.53:0/3273644494 <== osd.4 v2:172.21.15.155:6800/2046029108 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f98b4077140 con 0x7f98b4074670 2024-09-19T06:16:02.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.448+0000 7f98d0d2e640 1 -- 172.21.15.53:0/3273644494 --> [v2:172.21.15.155:6800/2046029108,v1:172.21.15.155:6801/2046029108] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f98cc062900 con 0x7f98b4074670 2024-09-19T06:16:02.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.449+0000 7f98bb7fe640 1 -- 172.21.15.53:0/3273644494 <== osd.4 v2:172.21.15.155:6800/2046029108 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f98cc062900 con 0x7f98b4074670 2024-09-19T06:16:02.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.449+0000 7f98d0d2e640 1 -- 172.21.15.53:0/3273644494 >> [v2:172.21.15.155:6800/2046029108,v1:172.21.15.155:6801/2046029108] conn(0x7f98b4074670 msgr2=0x7f98b4076a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:02.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.449+0000 7f98d0d2e640 1 --2- 172.21.15.53:0/3273644494 >> [v2:172.21.15.155:6800/2046029108,v1:172.21.15.155:6801/2046029108] conn(0x7f98b4074670 0x7f98b4076a90 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.449+0000 7f98d0d2e640 1 -- 172.21.15.53:0/3273644494 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f98b406e2b0 msgr2=0x7f98b4070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:02.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.449+0000 7f98d0d2e640 1 --2- 172.21.15.53:0/3273644494 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f98b406e2b0 0x7f98b4070770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f98cc199050 tx=0x7f98c0005a20 comp rx=0 tx=0).stop 2024-09-19T06:16:02.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.449+0000 7f98d0d2e640 1 -- 172.21.15.53:0/3273644494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98cc101900 msgr2=0x7f98cc19d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:02.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.449+0000 7f98d0d2e640 1 --2- 172.21.15.53:0/3273644494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98cc101900 0x7f98cc19d6a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00b420 tx=0x7f98bc00b8e0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.449+0000 7f98d0d2e640 1 -- 172.21.15.53:0/3273644494 shutdown_connections 2024-09-19T06:16:02.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.449+0000 7f98d0d2e640 1 --2- 172.21.15.53:0/3273644494 >> [v2:172.21.15.155:6800/2046029108,v1:172.21.15.155:6801/2046029108] conn(0x7f98b4074670 0x7f98b4076a90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.449+0000 7f98d0d2e640 1 --2- 172.21.15.53:0/3273644494 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f98b406e2b0 0x7f98b4070770 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.449+0000 7f98d0d2e640 1 --2- 172.21.15.53:0/3273644494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98cc107900 0x7f98cc19dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.449+0000 7f98d0d2e640 1 --2- 172.21.15.53:0/3273644494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98cc101900 0x7f98cc19d6a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.449+0000 7f98d0d2e640 1 -- 172.21.15.53:0/3273644494 >> 172.21.15.53:0/3273644494 conn(0x7f98cc0fd600 msgr2=0x7f98cc0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:02.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.449+0000 7f98d0d2e640 1 -- 172.21.15.53:0/3273644494 shutdown_connections 2024-09-19T06:16:02.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:02.450+0000 7f98d0d2e640 1 -- 172.21.15.53:0/3273644494 wait complete. 2024-09-19T06:16:02.987 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:02 smithi053 ceph-mon[27968]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-19T06:16:03.014 INFO:teuthology.orchestra.run.smithi053.stdout:81604378655 2024-09-19T06:16:03.014 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-19T06:16:03.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:02 smithi155 ceph-mon[35292]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-19T06:16:03.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.688+0000 7f77ba054640 1 -- 172.21.15.53:0/692913715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77b41033e0 msgr2=0x7f77b410b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:03.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.688+0000 7f77ba054640 1 --2- 172.21.15.53:0/692913715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77b41033e0 0x7f77b410b8d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f77a8009880 tx=0x7f77a802a530 comp rx=0 tx=0).stop 2024-09-19T06:16:03.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.688+0000 7f77ba054640 1 -- 172.21.15.53:0/692913715 shutdown_connections 2024-09-19T06:16:03.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.688+0000 7f77ba054640 1 --2- 172.21.15.53:0/692913715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77b41033e0 0x7f77b410b8d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:03.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.688+0000 7f77ba054640 1 --2- 172.21.15.53:0/692913715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77b4102ac0 0x7f77b4102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:03.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.688+0000 7f77ba054640 1 -- 172.21.15.53:0/692913715 >> 172.21.15.53:0/692913715 conn(0x7f77b40fc520 msgr2=0x7f77b40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:03.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.688+0000 7f77ba054640 1 -- 172.21.15.53:0/692913715 shutdown_connections 2024-09-19T06:16:03.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.688+0000 7f77ba054640 1 -- 172.21.15.53:0/692913715 wait complete. 2024-09-19T06:16:03.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.689+0000 7f77ba054640 1 Processor -- start 2024-09-19T06:16:03.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.689+0000 7f77ba054640 1 -- start start 2024-09-19T06:16:03.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.690+0000 7f77ba054640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77b4102ac0 0x7f77b419d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:03.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.690+0000 7f77ba054640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77b41033e0 0x7f77b419dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:03.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.690+0000 7f77ba054640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f77b410edc0 con 0x7f77b41033e0 2024-09-19T06:16:03.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.690+0000 7f77ba054640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f77b410ec40 con 0x7f77b4102ac0 2024-09-19T06:16:03.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.690+0000 7f77b2ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77b41033e0 0x7f77b419dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:03.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.690+0000 7f77b2ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77b41033e0 0x7f77b419dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41392/0 (socket says 172.21.15.53:41392) 2024-09-19T06:16:03.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.690+0000 7f77b2ffd640 1 -- 172.21.15.53:0/1163619597 learned_addr learned my addr 172.21.15.53:0/1163619597 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:03.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.690+0000 7f77b37fe640 1 --2- 172.21.15.53:0/1163619597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77b4102ac0 0x7f77b419d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:03.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.690+0000 7f77b2ffd640 1 -- 172.21.15.53:0/1163619597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77b4102ac0 msgr2=0x7f77b419d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:03.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.690+0000 7f77b2ffd640 1 --2- 172.21.15.53:0/1163619597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77b4102ac0 0x7f77b419d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:03.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.690+0000 7f77b2ffd640 1 -- 172.21.15.53:0/1163619597 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b4197820 con 0x7f77b41033e0 2024-09-19T06:16:03.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.690+0000 7f77b2ffd640 1 --2- 172.21.15.53:0/1163619597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77b41033e0 0x7f77b419dbe0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f77a8006010 tx=0x7f77a80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:03.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.690+0000 7f77b0ff9640 1 -- 172.21.15.53:0/1163619597 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f77a803f070 con 0x7f77b41033e0 2024-09-19T06:16:03.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.690+0000 7f77b0ff9640 1 -- 172.21.15.53:0/1163619597 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77a8004b10 con 0x7f77b41033e0 2024-09-19T06:16:03.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.690+0000 7f77ba054640 1 -- 172.21.15.53:0/1163619597 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77b4197ab0 con 0x7f77b41033e0 2024-09-19T06:16:03.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.690+0000 7f77b0ff9640 1 -- 172.21.15.53:0/1163619597 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f77a8035030 con 0x7f77b41033e0 2024-09-19T06:16:03.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.690+0000 7f77ba054640 1 -- 172.21.15.53:0/1163619597 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77b4197fe0 con 0x7f77b41033e0 2024-09-19T06:16:03.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.691+0000 7f77ba054640 1 -- 172.21.15.53:0/1163619597 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f77b41985a0 con 0x7f77b41033e0 2024-09-19T06:16:03.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.692+0000 7f77b0ff9640 1 -- 172.21.15.53:0/1163619597 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f77a8042070 con 0x7f77b41033e0 2024-09-19T06:16:03.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.693+0000 7f77b0ff9640 1 --2- 172.21.15.53:0/1163619597 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f779c06e300 0x7f779c0707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:03.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.693+0000 7f77b0ff9640 1 -- 172.21.15.53:0/1163619597 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5502+0+0 (secure 0 0 0) 0x7f77a80af080 con 0x7f77b41033e0 2024-09-19T06:16:03.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.693+0000 7f77b37fe640 1 --2- 172.21.15.53:0/1163619597 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f779c06e300 0x7f779c0707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:03.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.693+0000 7f77b0ff9640 1 --2- 172.21.15.53:0/1163619597 >> [v2:172.21.15.155:6824/2269161534,v1:172.21.15.155:6825/2269161534] conn(0x7f779c0746c0 0x7f779c076ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:03.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.693+0000 7f77b0ff9640 1 -- 172.21.15.53:0/1163619597 --> [v2:172.21.15.155:6824/2269161534,v1:172.21.15.155:6825/2269161534] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f77a80054d0 con 0x7f779c0746c0 2024-09-19T06:16:03.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.693+0000 7f77b0ff9640 1 -- 172.21.15.53:0/1163619597 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f77a80af520 con 0x7f77b41033e0 2024-09-19T06:16:03.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.693+0000 7f77b37fe640 1 --2- 172.21.15.53:0/1163619597 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f779c06e300 0x7f779c0707c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f77a4009890 tx=0x7f77a4008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:03.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.693+0000 7f77b3fff640 1 --2- 172.21.15.53:0/1163619597 >> [v2:172.21.15.155:6824/2269161534,v1:172.21.15.155:6825/2269161534] conn(0x7f779c0746c0 0x7f779c076ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:03.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.694+0000 7f77b3fff640 1 --2- 172.21.15.53:0/1163619597 >> [v2:172.21.15.155:6824/2269161534,v1:172.21.15.155:6825/2269161534] conn(0x7f779c0746c0 0x7f779c076ae0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:03.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.694+0000 7f77b0ff9640 1 -- 172.21.15.53:0/1163619597 <== osd.7 v2:172.21.15.155:6824/2269161534 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f77a80054d0 con 0x7f779c0746c0 2024-09-19T06:16:03.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.708+0000 7f77ba054640 1 -- 172.21.15.53:0/1163619597 --> [v2:172.21.15.155:6824/2269161534,v1:172.21.15.155:6825/2269161534] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f77b4062880 con 0x7f779c0746c0 2024-09-19T06:16:03.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.709+0000 7f77b0ff9640 1 -- 172.21.15.53:0/1163619597 <== osd.7 v2:172.21.15.155:6824/2269161534 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f77b4062880 con 0x7f779c0746c0 2024-09-19T06:16:03.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.709+0000 7f77ba054640 1 -- 172.21.15.53:0/1163619597 >> [v2:172.21.15.155:6824/2269161534,v1:172.21.15.155:6825/2269161534] conn(0x7f779c0746c0 msgr2=0x7f779c076ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:03.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.709+0000 7f77ba054640 1 --2- 172.21.15.53:0/1163619597 >> [v2:172.21.15.155:6824/2269161534,v1:172.21.15.155:6825/2269161534] conn(0x7f779c0746c0 0x7f779c076ae0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:03.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.709+0000 7f77ba054640 1 -- 172.21.15.53:0/1163619597 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f779c06e300 msgr2=0x7f779c0707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:03.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.709+0000 7f77ba054640 1 --2- 172.21.15.53:0/1163619597 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f779c06e300 0x7f779c0707c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f77a4009890 tx=0x7f77a4008040 comp rx=0 tx=0).stop 2024-09-19T06:16:03.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.709+0000 7f77ba054640 1 -- 172.21.15.53:0/1163619597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77b41033e0 msgr2=0x7f77b419dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:03.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.709+0000 7f77ba054640 1 --2- 172.21.15.53:0/1163619597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77b41033e0 0x7f77b419dbe0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f77a8006010 tx=0x7f77a80045e0 comp rx=0 tx=0).stop 2024-09-19T06:16:03.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.709+0000 7f77ba054640 1 -- 172.21.15.53:0/1163619597 shutdown_connections 2024-09-19T06:16:03.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.709+0000 7f77ba054640 1 --2- 172.21.15.53:0/1163619597 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f779c06e300 0x7f779c0707c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:03.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.709+0000 7f77ba054640 1 --2- 172.21.15.53:0/1163619597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77b41033e0 0x7f77b419dbe0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:03.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.709+0000 7f77ba054640 1 --2- 172.21.15.53:0/1163619597 >> [v2:172.21.15.155:6824/2269161534,v1:172.21.15.155:6825/2269161534] conn(0x7f779c0746c0 0x7f779c076ae0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:03.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.709+0000 7f77ba054640 1 --2- 172.21.15.53:0/1163619597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77b4102ac0 0x7f77b419d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:03.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.709+0000 7f77ba054640 1 -- 172.21.15.53:0/1163619597 >> 172.21.15.53:0/1163619597 conn(0x7f77b40fc520 msgr2=0x7f77b4103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:03.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.709+0000 7f77ba054640 1 -- 172.21.15.53:0/1163619597 shutdown_connections 2024-09-19T06:16:03.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:03.709+0000 7f77ba054640 1 -- 172.21.15.53:0/1163619597 wait complete. 2024-09-19T06:16:04.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.100+0000 7f37612e2640 1 -- 172.21.15.53:0/886532267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f375c106800 msgr2=0x7f375c106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:04.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.100+0000 7f37612e2640 1 --2- 172.21.15.53:0/886532267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f375c106800 0x7f375c106be0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f374c009880 tx=0x7f374c02a4a0 comp rx=0 tx=0).stop 2024-09-19T06:16:04.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.100+0000 7f37612e2640 1 -- 172.21.15.53:0/886532267 shutdown_connections 2024-09-19T06:16:04.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.100+0000 7f37612e2640 1 --2- 172.21.15.53:0/886532267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f375c100800 0x7f375c100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:04.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.100+0000 7f37612e2640 1 --2- 172.21.15.53:0/886532267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f375c106800 0x7f375c106be0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:04.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.100+0000 7f37612e2640 1 -- 172.21.15.53:0/886532267 >> 172.21.15.53:0/886532267 conn(0x7f375c0fc520 msgr2=0x7f375c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:04.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.100+0000 7f37612e2640 1 -- 172.21.15.53:0/886532267 shutdown_connections 2024-09-19T06:16:04.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.100+0000 7f37612e2640 1 -- 172.21.15.53:0/886532267 wait complete. 2024-09-19T06:16:04.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.100+0000 7f37612e2640 1 Processor -- start 2024-09-19T06:16:04.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.100+0000 7f37612e2640 1 -- start start 2024-09-19T06:16:04.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.101+0000 7f37612e2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f375c100800 0x7f375c19f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:04.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.101+0000 7f37612e2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f375c106800 0x7f375c19fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:04.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.101+0000 7f37612e2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f375c10dca0 con 0x7f375c100800 2024-09-19T06:16:04.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.101+0000 7f37612e2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f375c10db20 con 0x7f375c106800 2024-09-19T06:16:04.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.101+0000 7f375affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f375c100800 0x7f375c19f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:04.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.101+0000 7f375affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f375c100800 0x7f375c19f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41410/0 (socket says 172.21.15.53:41410) 2024-09-19T06:16:04.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.101+0000 7f375affd640 1 -- 172.21.15.53:0/1046303215 learned_addr learned my addr 172.21.15.53:0/1046303215 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:04.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.101+0000 7f375a7fc640 1 --2- 172.21.15.53:0/1046303215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f375c106800 0x7f375c19fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:04.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.101+0000 7f375affd640 1 -- 172.21.15.53:0/1046303215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f375c106800 msgr2=0x7f375c19fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:04.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.101+0000 7f375affd640 1 --2- 172.21.15.53:0/1046303215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f375c106800 0x7f375c19fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:04.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.101+0000 7f375affd640 1 -- 172.21.15.53:0/1046303215 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f375c1a0450 con 0x7f375c100800 2024-09-19T06:16:04.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.101+0000 7f375a7fc640 1 --2- 172.21.15.53:0/1046303215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f375c106800 0x7f375c19fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:16:04.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.101+0000 7f375affd640 1 --2- 172.21.15.53:0/1046303215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f375c100800 0x7f375c19f860 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f374c006010 tx=0x7f374c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:04.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.102+0000 7f3743fff640 1 -- 172.21.15.53:0/1046303215 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f374c03f070 con 0x7f375c100800 2024-09-19T06:16:04.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.102+0000 7f3743fff640 1 -- 172.21.15.53:0/1046303215 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f374c003aa0 con 0x7f375c100800 2024-09-19T06:16:04.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.102+0000 7f3743fff640 1 -- 172.21.15.53:0/1046303215 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f374c035030 con 0x7f375c100800 2024-09-19T06:16:04.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.102+0000 7f37612e2640 1 -- 172.21.15.53:0/1046303215 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f375c199ae0 con 0x7f375c100800 2024-09-19T06:16:04.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.102+0000 7f37612e2640 1 -- 172.21.15.53:0/1046303215 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f375c199fe0 con 0x7f375c100800 2024-09-19T06:16:04.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.102+0000 7f37612e2640 1 -- 172.21.15.53:0/1046303215 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f375c19a5a0 con 0x7f375c100800 2024-09-19T06:16:04.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.103+0000 7f3743fff640 1 -- 172.21.15.53:0/1046303215 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f374c042070 con 0x7f375c100800 2024-09-19T06:16:04.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.104+0000 7f3743fff640 1 --2- 172.21.15.53:0/1046303215 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f374406e2b0 0x7f3744070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:04.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.104+0000 7f375a7fc640 1 --2- 172.21.15.53:0/1046303215 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f374406e2b0 0x7f3744070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:04.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.104+0000 7f3743fff640 1 -- 172.21.15.53:0/1046303215 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5502+0+0 (secure 0 0 0) 0x7f374c0aeed0 con 0x7f375c100800 2024-09-19T06:16:04.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.104+0000 7f3743fff640 1 --2- 172.21.15.53:0/1046303215 >> [v2:172.21.15.53:6810/3638097004,v1:172.21.15.53:6811/3638097004] conn(0x7f3744074670 0x7f3744076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:04.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.104+0000 7f3743fff640 1 -- 172.21.15.53:0/1046303215 --> [v2:172.21.15.53:6810/3638097004,v1:172.21.15.53:6811/3638097004] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f374c001d30 con 0x7f3744074670 2024-09-19T06:16:04.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.104+0000 7f3743fff640 1 -- 172.21.15.53:0/1046303215 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f374c0b15e0 con 0x7f375c100800 2024-09-19T06:16:04.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.104+0000 7f375b7fe640 1 --2- 172.21.15.53:0/1046303215 >> [v2:172.21.15.53:6810/3638097004,v1:172.21.15.53:6811/3638097004] conn(0x7f3744074670 0x7f3744076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:04.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.104+0000 7f375a7fc640 1 --2- 172.21.15.53:0/1046303215 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f374406e2b0 0x7f3744070770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f375c0febb0 tx=0x7f3750008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:04.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.104+0000 7f375b7fe640 1 --2- 172.21.15.53:0/1046303215 >> [v2:172.21.15.53:6810/3638097004,v1:172.21.15.53:6811/3638097004] conn(0x7f3744074670 0x7f3744076a90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:04.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.105+0000 7f3743fff640 1 -- 172.21.15.53:0/1046303215 <== osd.1 v2:172.21.15.53:6810/3638097004 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f374c001d30 con 0x7f3744074670 2024-09-19T06:16:04.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.120+0000 7f37612e2640 1 -- 172.21.15.53:0/1046303215 --> [v2:172.21.15.53:6810/3638097004,v1:172.21.15.53:6811/3638097004] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f375c101f40 con 0x7f3744074670 2024-09-19T06:16:04.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.120+0000 7f3743fff640 1 -- 172.21.15.53:0/1046303215 <== osd.1 v2:172.21.15.53:6810/3638097004 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f375c101f40 con 0x7f3744074670 2024-09-19T06:16:04.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.121+0000 7f37612e2640 1 -- 172.21.15.53:0/1046303215 >> [v2:172.21.15.53:6810/3638097004,v1:172.21.15.53:6811/3638097004] conn(0x7f3744074670 msgr2=0x7f3744076a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:04.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.121+0000 7f37612e2640 1 --2- 172.21.15.53:0/1046303215 >> [v2:172.21.15.53:6810/3638097004,v1:172.21.15.53:6811/3638097004] conn(0x7f3744074670 0x7f3744076a90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:04.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.121+0000 7f37612e2640 1 -- 172.21.15.53:0/1046303215 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f374406e2b0 msgr2=0x7f3744070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:04.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.121+0000 7f37612e2640 1 --2- 172.21.15.53:0/1046303215 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f374406e2b0 0x7f3744070770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f375c0febb0 tx=0x7f3750008040 comp rx=0 tx=0).stop 2024-09-19T06:16:04.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.121+0000 7f37612e2640 1 -- 172.21.15.53:0/1046303215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f375c100800 msgr2=0x7f375c19f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:04.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.121+0000 7f37612e2640 1 --2- 172.21.15.53:0/1046303215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f375c100800 0x7f375c19f860 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f374c006010 tx=0x7f374c0048f0 comp rx=0 tx=0).stop 2024-09-19T06:16:04.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.121+0000 7f37612e2640 1 -- 172.21.15.53:0/1046303215 shutdown_connections 2024-09-19T06:16:04.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.121+0000 7f37612e2640 1 --2- 172.21.15.53:0/1046303215 >> [v2:172.21.15.53:6810/3638097004,v1:172.21.15.53:6811/3638097004] conn(0x7f3744074670 0x7f3744076a90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:04.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.121+0000 7f37612e2640 1 --2- 172.21.15.53:0/1046303215 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f374406e2b0 0x7f3744070770 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:04.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.121+0000 7f37612e2640 1 --2- 172.21.15.53:0/1046303215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f375c106800 0x7f375c19fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:04.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.121+0000 7f37612e2640 1 --2- 172.21.15.53:0/1046303215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f375c100800 0x7f375c19f860 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:04.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.121+0000 7f37612e2640 1 -- 172.21.15.53:0/1046303215 >> 172.21.15.53:0/1046303215 conn(0x7f375c0fc520 msgr2=0x7f375c0fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:04.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.121+0000 7f37612e2640 1 -- 172.21.15.53:0/1046303215 shutdown_connections 2024-09-19T06:16:04.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.121+0000 7f37612e2640 1 -- 172.21.15.53:0/1046303215 wait complete. 2024-09-19T06:16:04.507 INFO:teuthology.orchestra.run.smithi053.stdout:240518168583 2024-09-19T06:16:04.508 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-19T06:16:04.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.645+0000 7fb46eb79640 1 -- 172.21.15.53:0/817883295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4681067f0 msgr2=0x7fb468106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:04.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.645+0000 7fb46eb79640 1 --2- 172.21.15.53:0/817883295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4681067f0 0x7fb468106bd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb458009880 tx=0x7fb45802a4a0 comp rx=0 tx=0).stop 2024-09-19T06:16:04.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.645+0000 7fb46eb79640 1 -- 172.21.15.53:0/817883295 shutdown_connections 2024-09-19T06:16:04.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.645+0000 7fb46eb79640 1 --2- 172.21.15.53:0/817883295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4681007f0 0x7fb468100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:04.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.645+0000 7fb46eb79640 1 --2- 172.21.15.53:0/817883295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4681067f0 0x7fb468106bd0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:04.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.645+0000 7fb46eb79640 1 -- 172.21.15.53:0/817883295 >> 172.21.15.53:0/817883295 conn(0x7fb4680fc550 msgr2=0x7fb4680fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:04.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.645+0000 7fb46eb79640 1 -- 172.21.15.53:0/817883295 shutdown_connections 2024-09-19T06:16:04.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.645+0000 7fb46eb79640 1 -- 172.21.15.53:0/817883295 wait complete. 2024-09-19T06:16:04.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.645+0000 7fb46eb79640 1 Processor -- start 2024-09-19T06:16:04.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.645+0000 7fb46eb79640 1 -- start start 2024-09-19T06:16:04.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.645+0000 7fb46eb79640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4681007f0 0x7fb46819d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:04.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.645+0000 7fb46eb79640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4681067f0 0x7fb46819db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:04.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.646+0000 7fb46eb79640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb46810dc10 con 0x7fb4681067f0 2024-09-19T06:16:04.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.646+0000 7fb46eb79640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb46810da90 con 0x7fb4681007f0 2024-09-19T06:16:04.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.646+0000 7fb467fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4681067f0 0x7fb46819db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:04.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.646+0000 7fb467fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4681067f0 0x7fb46819db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41422/0 (socket says 172.21.15.53:41422) 2024-09-19T06:16:04.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.646+0000 7fb467fff640 1 -- 172.21.15.53:0/1891514529 learned_addr learned my addr 172.21.15.53:0/1891514529 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:04.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.646+0000 7fb46c915640 1 --2- 172.21.15.53:0/1891514529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4681007f0 0x7fb46819d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:04.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.646+0000 7fb467fff640 1 -- 172.21.15.53:0/1891514529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4681007f0 msgr2=0x7fb46819d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:04.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.646+0000 7fb467fff640 1 --2- 172.21.15.53:0/1891514529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4681007f0 0x7fb46819d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:04.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.646+0000 7fb467fff640 1 -- 172.21.15.53:0/1891514529 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4681976d0 con 0x7fb4681067f0 2024-09-19T06:16:04.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.646+0000 7fb467fff640 1 --2- 172.21.15.53:0/1891514529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4681067f0 0x7fb46819db40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb45000d8d0 tx=0x7fb45000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:04.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.646+0000 7fb465ffb640 1 -- 172.21.15.53:0/1891514529 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb450017070 con 0x7fb4681067f0 2024-09-19T06:16:04.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.646+0000 7fb465ffb640 1 -- 172.21.15.53:0/1891514529 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb450010460 con 0x7fb4681067f0 2024-09-19T06:16:04.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.646+0000 7fb46eb79640 1 -- 172.21.15.53:0/1891514529 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4681979c0 con 0x7fb4681067f0 2024-09-19T06:16:04.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.646+0000 7fb465ffb640 1 -- 172.21.15.53:0/1891514529 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb45000f5f0 con 0x7fb4681067f0 2024-09-19T06:16:04.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.647+0000 7fb46eb79640 1 -- 172.21.15.53:0/1891514529 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb468197f50 con 0x7fb4681067f0 2024-09-19T06:16:04.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.647+0000 7fb46eb79640 1 -- 172.21.15.53:0/1891514529 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb468198510 con 0x7fb4681067f0 2024-09-19T06:16:04.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.648+0000 7fb465ffb640 1 -- 172.21.15.53:0/1891514529 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7fb450010610 con 0x7fb4681067f0 2024-09-19T06:16:04.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.648+0000 7fb465ffb640 1 --2- 172.21.15.53:0/1891514529 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fb45406e2b0 0x7fb454070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:04.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.649+0000 7fb465ffb640 1 -- 172.21.15.53:0/1891514529 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5502+0+0 (secure 0 0 0) 0x7fb45008e000 con 0x7fb4681067f0 2024-09-19T06:16:04.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.649+0000 7fb46c915640 1 --2- 172.21.15.53:0/1891514529 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fb45406e2b0 0x7fb454070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:04.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.649+0000 7fb465ffb640 1 --2- 172.21.15.53:0/1891514529 >> [v2:172.21.15.53:6802/1456825097,v1:172.21.15.53:6803/1456825097] conn(0x7fb454074670 0x7fb454076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:04.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.649+0000 7fb465ffb640 1 -- 172.21.15.53:0/1891514529 --> [v2:172.21.15.53:6802/1456825097,v1:172.21.15.53:6803/1456825097] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb450004470 con 0x7fb454074670 2024-09-19T06:16:04.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.649+0000 7fb465ffb640 1 -- 172.21.15.53:0/1891514529 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fb450095050 con 0x7fb4681067f0 2024-09-19T06:16:04.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.649+0000 7fb46d116640 1 --2- 172.21.15.53:0/1891514529 >> [v2:172.21.15.53:6802/1456825097,v1:172.21.15.53:6803/1456825097] conn(0x7fb454074670 0x7fb454076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:04.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.649+0000 7fb46c915640 1 --2- 172.21.15.53:0/1891514529 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fb45406e2b0 0x7fb454070770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb458000c00 tx=0x7fb4580058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:04.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.649+0000 7fb46d116640 1 --2- 172.21.15.53:0/1891514529 >> [v2:172.21.15.53:6802/1456825097,v1:172.21.15.53:6803/1456825097] conn(0x7fb454074670 0x7fb454076a90 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:04.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.650+0000 7fb465ffb640 1 -- 172.21.15.53:0/1891514529 <== osd.0 v2:172.21.15.53:6802/1456825097 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb450004470 con 0x7fb454074670 2024-09-19T06:16:04.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.664+0000 7fb46eb79640 1 -- 172.21.15.53:0/1891514529 --> [v2:172.21.15.53:6802/1456825097,v1:172.21.15.53:6803/1456825097] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb46806a7e0 con 0x7fb454074670 2024-09-19T06:16:04.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.664+0000 7fb465ffb640 1 -- 172.21.15.53:0/1891514529 <== osd.0 v2:172.21.15.53:6802/1456825097 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb46806a7e0 con 0x7fb454074670 2024-09-19T06:16:04.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.664+0000 7fb46eb79640 1 -- 172.21.15.53:0/1891514529 >> [v2:172.21.15.53:6802/1456825097,v1:172.21.15.53:6803/1456825097] conn(0x7fb454074670 msgr2=0x7fb454076a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:04.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.664+0000 7fb46eb79640 1 --2- 172.21.15.53:0/1891514529 >> [v2:172.21.15.53:6802/1456825097,v1:172.21.15.53:6803/1456825097] conn(0x7fb454074670 0x7fb454076a90 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:04.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.664+0000 7fb46eb79640 1 -- 172.21.15.53:0/1891514529 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fb45406e2b0 msgr2=0x7fb454070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:04.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.664+0000 7fb46eb79640 1 --2- 172.21.15.53:0/1891514529 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fb45406e2b0 0x7fb454070770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb458000c00 tx=0x7fb4580058d0 comp rx=0 tx=0).stop 2024-09-19T06:16:04.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.664+0000 7fb46eb79640 1 -- 172.21.15.53:0/1891514529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4681067f0 msgr2=0x7fb46819db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:04.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.664+0000 7fb46eb79640 1 --2- 172.21.15.53:0/1891514529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4681067f0 0x7fb46819db40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb45000d8d0 tx=0x7fb45000dd90 comp rx=0 tx=0).stop 2024-09-19T06:16:04.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.665+0000 7fb46eb79640 1 -- 172.21.15.53:0/1891514529 shutdown_connections 2024-09-19T06:16:04.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.665+0000 7fb46eb79640 1 --2- 172.21.15.53:0/1891514529 >> [v2:172.21.15.53:6802/1456825097,v1:172.21.15.53:6803/1456825097] conn(0x7fb454074670 0x7fb454076a90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:04.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.665+0000 7fb46eb79640 1 --2- 172.21.15.53:0/1891514529 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fb45406e2b0 0x7fb454070770 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:04.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.665+0000 7fb46eb79640 1 --2- 172.21.15.53:0/1891514529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4681067f0 0x7fb46819db40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:04.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.665+0000 7fb46eb79640 1 --2- 172.21.15.53:0/1891514529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4681007f0 0x7fb46819d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:04.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.665+0000 7fb46eb79640 1 -- 172.21.15.53:0/1891514529 >> 172.21.15.53:0/1891514529 conn(0x7fb4680fc550 msgr2=0x7fb468069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:04.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.665+0000 7fb46eb79640 1 -- 172.21.15.53:0/1891514529 shutdown_connections 2024-09-19T06:16:04.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.665+0000 7fb46eb79640 1 -- 172.21.15.53:0/1891514529 wait complete. 2024-09-19T06:16:04.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.871+0000 7fc7648a4640 1 -- 172.21.15.53:0/262133600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc760107900 msgr2=0x7fc760107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:04.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.871+0000 7fc7648a4640 1 --2- 172.21.15.53:0/262133600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc760107900 0x7fc760107ce0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc750009c20 tx=0x7fc75002a970 comp rx=0 tx=0).stop 2024-09-19T06:16:04.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.872+0000 7fc7648a4640 1 -- 172.21.15.53:0/262133600 shutdown_connections 2024-09-19T06:16:04.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.872+0000 7fc7648a4640 1 --2- 172.21.15.53:0/262133600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc760101900 0x7fc760101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:04.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.872+0000 7fc7648a4640 1 --2- 172.21.15.53:0/262133600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc760107900 0x7fc760107ce0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:04.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.872+0000 7fc7648a4640 1 -- 172.21.15.53:0/262133600 >> 172.21.15.53:0/262133600 conn(0x7fc7600fd600 msgr2=0x7fc7600ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:04.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.872+0000 7fc7648a4640 1 -- 172.21.15.53:0/262133600 shutdown_connections 2024-09-19T06:16:04.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.872+0000 7fc7648a4640 1 -- 172.21.15.53:0/262133600 wait complete. 2024-09-19T06:16:04.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.872+0000 7fc7648a4640 1 Processor -- start 2024-09-19T06:16:04.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.872+0000 7fc7648a4640 1 -- start start 2024-09-19T06:16:04.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.873+0000 7fc7648a4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc760101900 0x7fc76019d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:04.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.873+0000 7fc7648a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc760107900 0x7fc76019db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:04.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.873+0000 7fc7648a4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc76010ed20 con 0x7fc760107900 2024-09-19T06:16:04.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.873+0000 7fc7648a4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc76010eba0 con 0x7fc760101900 2024-09-19T06:16:04.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.873+0000 7fc75dd9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc760107900 0x7fc76019db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:04.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.873+0000 7fc75dd9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc760107900 0x7fc76019db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41440/0 (socket says 172.21.15.53:41440) 2024-09-19T06:16:04.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.873+0000 7fc75dd9b640 1 -- 172.21.15.53:0/994875010 learned_addr learned my addr 172.21.15.53:0/994875010 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:04.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.873+0000 7fc75dd9b640 1 -- 172.21.15.53:0/994875010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc760101900 msgr2=0x7fc76019d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:04.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.873+0000 7fc75dd9b640 1 --2- 172.21.15.53:0/994875010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc760101900 0x7fc76019d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:04.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.873+0000 7fc75dd9b640 1 -- 172.21.15.53:0/994875010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc760197720 con 0x7fc760107900 2024-09-19T06:16:04.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.873+0000 7fc75dd9b640 1 --2- 172.21.15.53:0/994875010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc760107900 0x7fc76019db90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc75400d6f0 tx=0x7fc75400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:04.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.873+0000 7fc74f7fe640 1 -- 172.21.15.53:0/994875010 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc754017070 con 0x7fc760107900 2024-09-19T06:16:04.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.873+0000 7fc74f7fe640 1 -- 172.21.15.53:0/994875010 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc75400de00 con 0x7fc760107900 2024-09-19T06:16:04.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.873+0000 7fc7648a4640 1 -- 172.21.15.53:0/994875010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc760197a10 con 0x7fc760107900 2024-09-19T06:16:04.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.873+0000 7fc74f7fe640 1 -- 172.21.15.53:0/994875010 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc754004a60 con 0x7fc760107900 2024-09-19T06:16:04.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.873+0000 7fc7648a4640 1 -- 172.21.15.53:0/994875010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc760197fa0 con 0x7fc760107900 2024-09-19T06:16:04.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.874+0000 7fc7648a4640 1 -- 172.21.15.53:0/994875010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc760198560 con 0x7fc760107900 2024-09-19T06:16:04.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.875+0000 7fc74f7fe640 1 -- 172.21.15.53:0/994875010 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7fc754004d20 con 0x7fc760107900 2024-09-19T06:16:04.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.875+0000 7fc74f7fe640 1 --2- 172.21.15.53:0/994875010 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fc74806e2b0 0x7fc748070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:04.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.876+0000 7fc74f7fe640 1 -- 172.21.15.53:0/994875010 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5502+0+0 (secure 0 0 0) 0x7fc75408de00 con 0x7fc760107900 2024-09-19T06:16:04.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.876+0000 7fc75e59c640 1 --2- 172.21.15.53:0/994875010 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fc74806e2b0 0x7fc748070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:04.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.876+0000 7fc74f7fe640 1 --2- 172.21.15.53:0/994875010 >> [v2:172.21.15.155:6816/2574655677,v1:172.21.15.155:6817/2574655677] conn(0x7fc748074670 0x7fc748076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:04.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.876+0000 7fc74f7fe640 1 -- 172.21.15.53:0/994875010 --> [v2:172.21.15.155:6816/2574655677,v1:172.21.15.155:6817/2574655677] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc748077140 con 0x7fc748074670 2024-09-19T06:16:04.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.876+0000 7fc74f7fe640 1 -- 172.21.15.53:0/994875010 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fc754093520 con 0x7fc760107900 2024-09-19T06:16:04.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.876+0000 7fc75e59c640 1 --2- 172.21.15.53:0/994875010 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fc74806e2b0 0x7fc748070770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc750009430 tx=0x7fc75002f8c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:04.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.876+0000 7fc75ed9d640 1 --2- 172.21.15.53:0/994875010 >> [v2:172.21.15.155:6816/2574655677,v1:172.21.15.155:6817/2574655677] conn(0x7fc748074670 0x7fc748076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:04.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.877+0000 7fc75ed9d640 1 --2- 172.21.15.53:0/994875010 >> [v2:172.21.15.155:6816/2574655677,v1:172.21.15.155:6817/2574655677] conn(0x7fc748074670 0x7fc748076a90 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:04.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.877+0000 7fc74f7fe640 1 -- 172.21.15.53:0/994875010 <== osd.6 v2:172.21.15.155:6816/2574655677 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc748077140 con 0x7fc748074670 2024-09-19T06:16:04.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.891+0000 7fc7648a4640 1 -- 172.21.15.53:0/994875010 --> [v2:172.21.15.155:6816/2574655677,v1:172.21.15.155:6817/2574655677] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc7600628f0 con 0x7fc748074670 2024-09-19T06:16:04.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.892+0000 7fc74f7fe640 1 -- 172.21.15.53:0/994875010 <== osd.6 v2:172.21.15.155:6816/2574655677 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc7600628f0 con 0x7fc748074670 2024-09-19T06:16:04.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.892+0000 7fc7648a4640 1 -- 172.21.15.53:0/994875010 >> [v2:172.21.15.155:6816/2574655677,v1:172.21.15.155:6817/2574655677] conn(0x7fc748074670 msgr2=0x7fc748076a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:04.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.892+0000 7fc7648a4640 1 --2- 172.21.15.53:0/994875010 >> [v2:172.21.15.155:6816/2574655677,v1:172.21.15.155:6817/2574655677] conn(0x7fc748074670 0x7fc748076a90 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:04.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.892+0000 7fc7648a4640 1 -- 172.21.15.53:0/994875010 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fc74806e2b0 msgr2=0x7fc748070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:04.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.892+0000 7fc7648a4640 1 --2- 172.21.15.53:0/994875010 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fc74806e2b0 0x7fc748070770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc750009430 tx=0x7fc75002f8c0 comp rx=0 tx=0).stop 2024-09-19T06:16:04.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.892+0000 7fc7648a4640 1 -- 172.21.15.53:0/994875010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc760107900 msgr2=0x7fc76019db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:04.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.892+0000 7fc7648a4640 1 --2- 172.21.15.53:0/994875010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc760107900 0x7fc76019db90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc75400d6f0 tx=0x7fc75400dbb0 comp rx=0 tx=0).stop 2024-09-19T06:16:04.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.892+0000 7fc7648a4640 1 -- 172.21.15.53:0/994875010 shutdown_connections 2024-09-19T06:16:04.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.892+0000 7fc7648a4640 1 --2- 172.21.15.53:0/994875010 >> [v2:172.21.15.155:6816/2574655677,v1:172.21.15.155:6817/2574655677] conn(0x7fc748074670 0x7fc748076a90 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:04.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.892+0000 7fc7648a4640 1 --2- 172.21.15.53:0/994875010 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fc74806e2b0 0x7fc748070770 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:04.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.892+0000 7fc7648a4640 1 --2- 172.21.15.53:0/994875010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc760107900 0x7fc76019db90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:04.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.892+0000 7fc7648a4640 1 --2- 172.21.15.53:0/994875010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc760101900 0x7fc76019d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:04.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.892+0000 7fc7648a4640 1 -- 172.21.15.53:0/994875010 >> 172.21.15.53:0/994875010 conn(0x7fc7600fd600 msgr2=0x7fc7600fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:04.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.892+0000 7fc7648a4640 1 -- 172.21.15.53:0/994875010 shutdown_connections 2024-09-19T06:16:04.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:04.892+0000 7fc7648a4640 1 -- 172.21.15.53:0/994875010 wait complete. 2024-09-19T06:16:05.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:04 smithi053 ceph-mon[27968]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T06:16:05.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:04 smithi155 ceph-mon[35292]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T06:16:05.490 INFO:teuthology.orchestra.run.smithi053.stdout:133143986197 2024-09-19T06:16:05.490 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-19T06:16:06.137 INFO:teuthology.orchestra.run.smithi053.stdout:60129542182 2024-09-19T06:16:06.137 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-19T06:16:06.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.430+0000 7f2b9d4ac640 1 -- 172.21.15.53:0/3958932498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b98101900 msgr2=0x7f2b98101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:06.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.430+0000 7f2b9d4ac640 1 --2- 172.21.15.53:0/3958932498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b98101900 0x7f2b98101d60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2b80009880 tx=0x7f2b8002a4a0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.430+0000 7f2b9d4ac640 1 -- 172.21.15.53:0/3958932498 shutdown_connections 2024-09-19T06:16:06.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.430+0000 7f2b9d4ac640 1 --2- 172.21.15.53:0/3958932498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b98101900 0x7f2b98101d60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.430+0000 7f2b9d4ac640 1 --2- 172.21.15.53:0/3958932498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b98107900 0x7f2b98107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.430+0000 7f2b9d4ac640 1 -- 172.21.15.53:0/3958932498 >> 172.21.15.53:0/3958932498 conn(0x7f2b980fd620 msgr2=0x7f2b980ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:06.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.430+0000 7f2b9d4ac640 1 -- 172.21.15.53:0/3958932498 shutdown_connections 2024-09-19T06:16:06.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.430+0000 7f2b9d4ac640 1 -- 172.21.15.53:0/3958932498 wait complete. 2024-09-19T06:16:06.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.431+0000 7f2b9d4ac640 1 Processor -- start 2024-09-19T06:16:06.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.431+0000 7f2b9d4ac640 1 -- start start 2024-09-19T06:16:06.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.431+0000 7f2b9d4ac640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b98101900 0x7f2b9819d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:06.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.431+0000 7f2b9d4ac640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b98107900 0x7f2b9819dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:06.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.431+0000 7f2b9d4ac640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9810eda0 con 0x7f2b98107900 2024-09-19T06:16:06.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.431+0000 7f2b9d4ac640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9810ec20 con 0x7f2b98101900 2024-09-19T06:16:06.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.432+0000 7f2b967fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b98107900 0x7f2b9819dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:06.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.432+0000 7f2b967fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b98107900 0x7f2b9819dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41464/0 (socket says 172.21.15.53:41464) 2024-09-19T06:16:06.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.432+0000 7f2b967fc640 1 -- 172.21.15.53:0/3553731447 learned_addr learned my addr 172.21.15.53:0/3553731447 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:06.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.432+0000 7f2b96ffd640 1 --2- 172.21.15.53:0/3553731447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b98101900 0x7f2b9819d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:06.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.432+0000 7f2b967fc640 1 -- 172.21.15.53:0/3553731447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b98101900 msgr2=0x7f2b9819d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:06.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.432+0000 7f2b967fc640 1 --2- 172.21.15.53:0/3553731447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b98101900 0x7f2b9819d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.432+0000 7f2b967fc640 1 -- 172.21.15.53:0/3553731447 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b98197820 con 0x7f2b98107900 2024-09-19T06:16:06.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.432+0000 7f2b967fc640 1 --2- 172.21.15.53:0/3553731447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b98107900 0x7f2b9819dbe0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2b80006010 tx=0x7f2b800048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:06.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.432+0000 7f2b8ffff640 1 -- 172.21.15.53:0/3553731447 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2b8003f070 con 0x7f2b98107900 2024-09-19T06:16:06.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.432+0000 7f2b9d4ac640 1 -- 172.21.15.53:0/3553731447 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b98197ab0 con 0x7f2b98107900 2024-09-19T06:16:06.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.432+0000 7f2b8ffff640 1 -- 172.21.15.53:0/3553731447 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b80003aa0 con 0x7f2b98107900 2024-09-19T06:16:06.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.432+0000 7f2b8ffff640 1 -- 172.21.15.53:0/3553731447 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2b80035030 con 0x7f2b98107900 2024-09-19T06:16:06.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.432+0000 7f2b9d4ac640 1 -- 172.21.15.53:0/3553731447 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b98197fe0 con 0x7f2b98107900 2024-09-19T06:16:06.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.433+0000 7f2b9d4ac640 1 -- 172.21.15.53:0/3553731447 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2b98198660 con 0x7f2b98107900 2024-09-19T06:16:06.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.434+0000 7f2b8ffff640 1 -- 172.21.15.53:0/3553731447 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f2b80042070 con 0x7f2b98107900 2024-09-19T06:16:06.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.434+0000 7f2b8ffff640 1 --2- 172.21.15.53:0/3553731447 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f2b8406e2b0 0x7f2b84070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:06.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.434+0000 7f2b8ffff640 1 -- 172.21.15.53:0/3553731447 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5502+0+0 (secure 0 0 0) 0x7f2b800af470 con 0x7f2b98107900 2024-09-19T06:16:06.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.434+0000 7f2b96ffd640 1 --2- 172.21.15.53:0/3553731447 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f2b8406e2b0 0x7f2b84070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:06.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.434+0000 7f2b8ffff640 1 --2- 172.21.15.53:0/3553731447 >> [v2:172.21.15.155:6808/464507356,v1:172.21.15.155:6809/464507356] conn(0x7f2b84074670 0x7f2b84076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:06.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.434+0000 7f2b8ffff640 1 -- 172.21.15.53:0/3553731447 --> [v2:172.21.15.155:6808/464507356,v1:172.21.15.155:6809/464507356] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2b8004de50 con 0x7f2b84074670 2024-09-19T06:16:06.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.435+0000 7f2b8ffff640 1 -- 172.21.15.53:0/3553731447 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f2b800af870 con 0x7f2b98107900 2024-09-19T06:16:06.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.435+0000 7f2b977fe640 1 --2- 172.21.15.53:0/3553731447 >> [v2:172.21.15.155:6808/464507356,v1:172.21.15.155:6809/464507356] conn(0x7f2b84074670 0x7f2b84076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:06.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.435+0000 7f2b96ffd640 1 --2- 172.21.15.53:0/3553731447 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f2b8406e2b0 0x7f2b84070770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2b90009890 tx=0x7f2b90008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:06.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.435+0000 7f2b977fe640 1 --2- 172.21.15.53:0/3553731447 >> [v2:172.21.15.155:6808/464507356,v1:172.21.15.155:6809/464507356] conn(0x7f2b84074670 0x7f2b84076a90 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:06.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.436+0000 7f2b8ffff640 1 -- 172.21.15.53:0/3553731447 <== osd.5 v2:172.21.15.155:6808/464507356 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2b8004de50 con 0x7f2b84074670 2024-09-19T06:16:06.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.450+0000 7f2b9d4ac640 1 -- 172.21.15.53:0/3553731447 --> [v2:172.21.15.155:6808/464507356,v1:172.21.15.155:6809/464507356] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2b98103040 con 0x7f2b84074670 2024-09-19T06:16:06.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.450+0000 7f2b8ffff640 1 -- 172.21.15.53:0/3553731447 <== osd.5 v2:172.21.15.155:6808/464507356 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2b98103040 con 0x7f2b84074670 2024-09-19T06:16:06.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.450+0000 7f2b9d4ac640 1 -- 172.21.15.53:0/3553731447 >> [v2:172.21.15.155:6808/464507356,v1:172.21.15.155:6809/464507356] conn(0x7f2b84074670 msgr2=0x7f2b84076a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:06.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.450+0000 7f2b9d4ac640 1 --2- 172.21.15.53:0/3553731447 >> [v2:172.21.15.155:6808/464507356,v1:172.21.15.155:6809/464507356] conn(0x7f2b84074670 0x7f2b84076a90 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.451+0000 7f2b9d4ac640 1 -- 172.21.15.53:0/3553731447 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f2b8406e2b0 msgr2=0x7f2b84070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:06.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.451+0000 7f2b9d4ac640 1 --2- 172.21.15.53:0/3553731447 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f2b8406e2b0 0x7f2b84070770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2b90009890 tx=0x7f2b90008040 comp rx=0 tx=0).stop 2024-09-19T06:16:06.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.451+0000 7f2b9d4ac640 1 -- 172.21.15.53:0/3553731447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b98107900 msgr2=0x7f2b9819dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:06.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.451+0000 7f2b9d4ac640 1 --2- 172.21.15.53:0/3553731447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b98107900 0x7f2b9819dbe0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2b80006010 tx=0x7f2b800048f0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.451+0000 7f2b9d4ac640 1 -- 172.21.15.53:0/3553731447 shutdown_connections 2024-09-19T06:16:06.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.451+0000 7f2b9d4ac640 1 --2- 172.21.15.53:0/3553731447 >> [v2:172.21.15.155:6808/464507356,v1:172.21.15.155:6809/464507356] conn(0x7f2b84074670 0x7f2b84076a90 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.451+0000 7f2b9d4ac640 1 --2- 172.21.15.53:0/3553731447 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f2b8406e2b0 0x7f2b84070770 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.451+0000 7f2b9d4ac640 1 --2- 172.21.15.53:0/3553731447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b98107900 0x7f2b9819dbe0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.451+0000 7f2b9d4ac640 1 --2- 172.21.15.53:0/3553731447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b98101900 0x7f2b9819d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.451+0000 7f2b9d4ac640 1 -- 172.21.15.53:0/3553731447 >> 172.21.15.53:0/3553731447 conn(0x7f2b980fd620 msgr2=0x7f2b980fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:06.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.451+0000 7f2b9d4ac640 1 -- 172.21.15.53:0/3553731447 shutdown_connections 2024-09-19T06:16:06.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.451+0000 7f2b9d4ac640 1 -- 172.21.15.53:0/3553731447 wait complete. 2024-09-19T06:16:06.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.640+0000 7fba95620640 1 -- 172.21.15.53:0/1527317710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba90101910 msgr2=0x7fba90101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:06.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.640+0000 7fba95620640 1 --2- 172.21.15.53:0/1527317710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba90101910 0x7fba90101d70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fba84009880 tx=0x7fba8402a4a0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.640+0000 7fba95620640 1 -- 172.21.15.53:0/1527317710 shutdown_connections 2024-09-19T06:16:06.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.640+0000 7fba95620640 1 --2- 172.21.15.53:0/1527317710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba90101910 0x7fba90101d70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.640+0000 7fba95620640 1 --2- 172.21.15.53:0/1527317710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba90107910 0x7fba90107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.640+0000 7fba95620640 1 -- 172.21.15.53:0/1527317710 >> 172.21.15.53:0/1527317710 conn(0x7fba900fd610 msgr2=0x7fba900ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:06.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.640+0000 7fba95620640 1 -- 172.21.15.53:0/1527317710 shutdown_connections 2024-09-19T06:16:06.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.640+0000 7fba95620640 1 -- 172.21.15.53:0/1527317710 wait complete. 2024-09-19T06:16:06.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.640+0000 7fba95620640 1 Processor -- start 2024-09-19T06:16:06.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.641+0000 7fba95620640 1 -- start start 2024-09-19T06:16:06.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.641+0000 7fba95620640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba90101910 0x7fba900744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:06.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.641+0000 7fba95620640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba90107910 0x7fba90074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:06.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.641+0000 7fba95620640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fba9010ed50 con 0x7fba90107910 2024-09-19T06:16:06.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.641+0000 7fba95620640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fba9010ebd0 con 0x7fba90101910 2024-09-19T06:16:06.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.641+0000 7fba8e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba90107910 0x7fba90074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:06.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.641+0000 7fba8e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba90107910 0x7fba90074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41484/0 (socket says 172.21.15.53:41484) 2024-09-19T06:16:06.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.641+0000 7fba8e7fc640 1 -- 172.21.15.53:0/2248569527 learned_addr learned my addr 172.21.15.53:0/2248569527 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:06.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.641+0000 7fba8effd640 1 --2- 172.21.15.53:0/2248569527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba90101910 0x7fba900744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:06.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.641+0000 7fba8e7fc640 1 -- 172.21.15.53:0/2248569527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba90101910 msgr2=0x7fba900744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:06.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.641+0000 7fba8e7fc640 1 --2- 172.21.15.53:0/2248569527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba90101910 0x7fba900744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.641+0000 7fba8e7fc640 1 -- 172.21.15.53:0/2248569527 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba90074f70 con 0x7fba90107910 2024-09-19T06:16:06.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.641+0000 7fba8effd640 1 --2- 172.21.15.53:0/2248569527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba90101910 0x7fba900744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:16:06.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.641+0000 7fba8e7fc640 1 --2- 172.21.15.53:0/2248569527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba90107910 0x7fba90074a30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fba8402a9c0 tx=0x7fba840045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:06.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.642+0000 7fba77fff640 1 -- 172.21.15.53:0/2248569527 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fba8403f070 con 0x7fba90107910 2024-09-19T06:16:06.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.642+0000 7fba77fff640 1 -- 172.21.15.53:0/2248569527 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba84004a50 con 0x7fba90107910 2024-09-19T06:16:06.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.642+0000 7fba95620640 1 -- 172.21.15.53:0/2248569527 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba90075200 con 0x7fba90107910 2024-09-19T06:16:06.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.642+0000 7fba77fff640 1 -- 172.21.15.53:0/2248569527 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fba84035030 con 0x7fba90107910 2024-09-19T06:16:06.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.642+0000 7fba95620640 1 -- 172.21.15.53:0/2248569527 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba901a3c90 con 0x7fba90107910 2024-09-19T06:16:06.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.643+0000 7fba95620640 1 -- 172.21.15.53:0/2248569527 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fba900628c0 con 0x7fba90107910 2024-09-19T06:16:06.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.643+0000 7fba77fff640 1 -- 172.21.15.53:0/2248569527 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7fba84042070 con 0x7fba90107910 2024-09-19T06:16:06.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.644+0000 7fba77fff640 1 --2- 172.21.15.53:0/2248569527 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fba7806e2b0 0x7fba78070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:06.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.644+0000 7fba77fff640 1 -- 172.21.15.53:0/2248569527 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5502+0+0 (secure 0 0 0) 0x7fba840aec60 con 0x7fba90107910 2024-09-19T06:16:06.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.644+0000 7fba8effd640 1 --2- 172.21.15.53:0/2248569527 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fba7806e2b0 0x7fba78070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:06.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.644+0000 7fba77fff640 1 --2- 172.21.15.53:0/2248569527 >> [v2:172.21.15.53:6826/2604012854,v1:172.21.15.53:6827/2604012854] conn(0x7fba78074670 0x7fba78076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:06.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.644+0000 7fba77fff640 1 -- 172.21.15.53:0/2248569527 --> [v2:172.21.15.53:6826/2604012854,v1:172.21.15.53:6827/2604012854] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fba840054d0 con 0x7fba78074670 2024-09-19T06:16:06.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.644+0000 7fba77fff640 1 -- 172.21.15.53:0/2248569527 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fba840b4380 con 0x7fba90107910 2024-09-19T06:16:06.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.644+0000 7fba8f7fe640 1 --2- 172.21.15.53:0/2248569527 >> [v2:172.21.15.53:6826/2604012854,v1:172.21.15.53:6827/2604012854] conn(0x7fba78074670 0x7fba78076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:06.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.644+0000 7fba8effd640 1 --2- 172.21.15.53:0/2248569527 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fba7806e2b0 0x7fba78070770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fba80009b50 tx=0x7fba80005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:06.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.645+0000 7fba8f7fe640 1 --2- 172.21.15.53:0/2248569527 >> [v2:172.21.15.53:6826/2604012854,v1:172.21.15.53:6827/2604012854] conn(0x7fba78074670 0x7fba78076a90 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:06.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.645+0000 7fba77fff640 1 -- 172.21.15.53:0/2248569527 <== osd.3 v2:172.21.15.53:6826/2604012854 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fba840054d0 con 0x7fba78074670 2024-09-19T06:16:06.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.659+0000 7fba95620640 1 -- 172.21.15.53:0/2248569527 --> [v2:172.21.15.53:6826/2604012854,v1:172.21.15.53:6827/2604012854] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fba90062cc0 con 0x7fba78074670 2024-09-19T06:16:06.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.659+0000 7fba77fff640 1 -- 172.21.15.53:0/2248569527 <== osd.3 v2:172.21.15.53:6826/2604012854 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fba90062cc0 con 0x7fba78074670 2024-09-19T06:16:06.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.659+0000 7fba95620640 1 -- 172.21.15.53:0/2248569527 >> [v2:172.21.15.53:6826/2604012854,v1:172.21.15.53:6827/2604012854] conn(0x7fba78074670 msgr2=0x7fba78076a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:06.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.659+0000 7fba95620640 1 --2- 172.21.15.53:0/2248569527 >> [v2:172.21.15.53:6826/2604012854,v1:172.21.15.53:6827/2604012854] conn(0x7fba78074670 0x7fba78076a90 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.659+0000 7fba95620640 1 -- 172.21.15.53:0/2248569527 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fba7806e2b0 msgr2=0x7fba78070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:06.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.659+0000 7fba95620640 1 --2- 172.21.15.53:0/2248569527 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fba7806e2b0 0x7fba78070770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fba80009b50 tx=0x7fba80005f90 comp rx=0 tx=0).stop 2024-09-19T06:16:06.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.659+0000 7fba95620640 1 -- 172.21.15.53:0/2248569527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba90107910 msgr2=0x7fba90074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:06.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.659+0000 7fba95620640 1 --2- 172.21.15.53:0/2248569527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba90107910 0x7fba90074a30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fba8402a9c0 tx=0x7fba840045e0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.659+0000 7fba95620640 1 -- 172.21.15.53:0/2248569527 shutdown_connections 2024-09-19T06:16:06.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.659+0000 7fba95620640 1 --2- 172.21.15.53:0/2248569527 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fba7806e2b0 0x7fba78070770 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.659+0000 7fba95620640 1 --2- 172.21.15.53:0/2248569527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba90107910 0x7fba90074a30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.659+0000 7fba95620640 1 --2- 172.21.15.53:0/2248569527 >> [v2:172.21.15.53:6826/2604012854,v1:172.21.15.53:6827/2604012854] conn(0x7fba78074670 0x7fba78076a90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.659+0000 7fba95620640 1 --2- 172.21.15.53:0/2248569527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba90101910 0x7fba900744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.659+0000 7fba95620640 1 -- 172.21.15.53:0/2248569527 >> 172.21.15.53:0/2248569527 conn(0x7fba900fd610 msgr2=0x7fba900fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:06.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.660+0000 7fba95620640 1 -- 172.21.15.53:0/2248569527 shutdown_connections 2024-09-19T06:16:06.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:06.660+0000 7fba95620640 1 -- 172.21.15.53:0/2248569527 wait complete. 2024-09-19T06:16:07.030 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:06 smithi053 ceph-mon[27968]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:07.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:06 smithi155 ceph-mon[35292]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:07.482 INFO:teuthology.orchestra.run.smithi053.stdout:111669149722 2024-09-19T06:16:07.482 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-19T06:16:08.086 INFO:teuthology.orchestra.run.smithi053.stdout:167503724560 2024-09-19T06:16:08.087 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-19T06:16:08.538 INFO:teuthology.orchestra.run.smithi053.stdout:201863462923 2024-09-19T06:16:08.539 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-19T06:16:08.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:08 smithi155 ceph-mon[35292]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:09.051 INFO:teuthology.orchestra.run.smithi053.stdout:38654705707 2024-09-19T06:16:09.051 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-19T06:16:09.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:08 smithi053 ceph-mon[27968]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:09.147 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:09.149 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:09.150 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:09.151 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:09.193 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:10.049 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:09 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/1040375610' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T06:16:10.228 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:09 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/1040375610' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T06:16:10.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.728+0000 7fa5c9a29640 1 -- 172.21.15.53:0/1397776085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5c4107900 msgr2=0x7fa5c4107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:10.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.728+0000 7fa5c9a29640 1 --2- 172.21.15.53:0/1397776085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5c4107900 0x7fa5c4107ce0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009880 tx=0x7fa5b402a4a0 comp rx=0 tx=0).stop 2024-09-19T06:16:10.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.728+0000 7fa5c9a29640 1 -- 172.21.15.53:0/1397776085 shutdown_connections 2024-09-19T06:16:10.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.728+0000 7fa5c9a29640 1 --2- 172.21.15.53:0/1397776085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5c4101900 0x7fa5c4101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:10.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.728+0000 7fa5c9a29640 1 --2- 172.21.15.53:0/1397776085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5c4107900 0x7fa5c4107ce0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:10.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.728+0000 7fa5c9a29640 1 -- 172.21.15.53:0/1397776085 >> 172.21.15.53:0/1397776085 conn(0x7fa5c40fd600 msgr2=0x7fa5c40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:10.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.728+0000 7fa5c9a29640 1 -- 172.21.15.53:0/1397776085 shutdown_connections 2024-09-19T06:16:10.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.728+0000 7fa5c9a29640 1 -- 172.21.15.53:0/1397776085 wait complete. 2024-09-19T06:16:10.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.729+0000 7fa5c9a29640 1 Processor -- start 2024-09-19T06:16:10.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.729+0000 7fa5c9a29640 1 -- start start 2024-09-19T06:16:10.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.730+0000 7fa5c9a29640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5c4101900 0x7fa5c419d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:10.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.730+0000 7fa5c9a29640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5c4107900 0x7fa5c419dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:10.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.730+0000 7fa5c9a29640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c410ed40 con 0x7fa5c4101900 2024-09-19T06:16:10.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.730+0000 7fa5c9a29640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c410ebc0 con 0x7fa5c4107900 2024-09-19T06:16:10.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.730+0000 7fa5c2ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5c4101900 0x7fa5c419d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:10.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.730+0000 7fa5c2ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5c4101900 0x7fa5c419d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41506/0 (socket says 172.21.15.53:41506) 2024-09-19T06:16:10.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.730+0000 7fa5c2ffd640 1 -- 172.21.15.53:0/1040375610 learned_addr learned my addr 172.21.15.53:0/1040375610 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:10.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.730+0000 7fa5c27fc640 1 --2- 172.21.15.53:0/1040375610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5c4107900 0x7fa5c419dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:10.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.730+0000 7fa5c2ffd640 1 -- 172.21.15.53:0/1040375610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5c4107900 msgr2=0x7fa5c419dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:10.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.730+0000 7fa5c2ffd640 1 --2- 172.21.15.53:0/1040375610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5c4107900 0x7fa5c419dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:10.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.730+0000 7fa5c2ffd640 1 -- 172.21.15.53:0/1040375610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c4197740 con 0x7fa5c4101900 2024-09-19T06:16:10.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.730+0000 7fa5c2ffd640 1 --2- 172.21.15.53:0/1040375610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5c4101900 0x7fa5c419d670 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa5b402a9c0 tx=0x7fa5b40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:10.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.730+0000 7fa5c8a27640 1 -- 172.21.15.53:0/1040375610 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa5b403f070 con 0x7fa5c4101900 2024-09-19T06:16:10.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.730+0000 7fa5c9a29640 1 -- 172.21.15.53:0/1040375610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c41979d0 con 0x7fa5c4101900 2024-09-19T06:16:10.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.730+0000 7fa5c8a27640 1 -- 172.21.15.53:0/1040375610 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5b4004a50 con 0x7fa5c4101900 2024-09-19T06:16:10.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.730+0000 7fa5c8a27640 1 -- 172.21.15.53:0/1040375610 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa5b4035030 con 0x7fa5c4101900 2024-09-19T06:16:10.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.730+0000 7fa5c9a29640 1 -- 172.21.15.53:0/1040375610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c4197f00 con 0x7fa5c4101900 2024-09-19T06:16:10.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.731+0000 7fa5c9a29640 1 -- 172.21.15.53:0/1040375610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c4103040 con 0x7fa5c4101900 2024-09-19T06:16:10.274 INFO:teuthology.orchestra.run.smithi053.stdout:81604378656 2024-09-19T06:16:10.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.732+0000 7fa5c8a27640 1 -- 172.21.15.53:0/1040375610 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7fa5b4042070 con 0x7fa5c4101900 2024-09-19T06:16:10.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.732+0000 7fa5c8a27640 1 --2- 172.21.15.53:0/1040375610 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fa5ac06e2b0 0x7fa5ac070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:10.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.732+0000 7fa5c8a27640 1 -- 172.21.15.53:0/1040375610 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5502+0+0 (secure 0 0 0) 0x7fa5b40aebb0 con 0x7fa5c4101900 2024-09-19T06:16:10.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.732+0000 7fa5c27fc640 1 --2- 172.21.15.53:0/1040375610 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fa5ac06e2b0 0x7fa5ac070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:10.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.733+0000 7fa5c27fc640 1 --2- 172.21.15.53:0/1040375610 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fa5ac06e2b0 0x7fa5ac070770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4198f90 tx=0x7fa5b8005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:10.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.737+0000 7fa5c8a27640 1 -- 172.21.15.53:0/1040375610 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa5b40b5080 con 0x7fa5c4101900 2024-09-19T06:16:10.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.918+0000 7fa5c9a29640 1 -- 172.21.15.53:0/1040375610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fa5c4109e70 con 0x7fa5c4101900 2024-09-19T06:16:10.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.918+0000 7fa5c8a27640 1 -- 172.21.15.53:0/1040375610 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa5b4080110 con 0x7fa5c4101900 2024-09-19T06:16:10.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.920+0000 7fa5c9a29640 1 -- 172.21.15.53:0/1040375610 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fa5ac06e2b0 msgr2=0x7fa5ac070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:10.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.920+0000 7fa5c9a29640 1 --2- 172.21.15.53:0/1040375610 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fa5ac06e2b0 0x7fa5ac070770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4198f90 tx=0x7fa5b8005f90 comp rx=0 tx=0).stop 2024-09-19T06:16:10.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.920+0000 7fa5c9a29640 1 -- 172.21.15.53:0/1040375610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5c4101900 msgr2=0x7fa5c419d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:10.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.920+0000 7fa5c9a29640 1 --2- 172.21.15.53:0/1040375610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5c4101900 0x7fa5c419d670 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa5b402a9c0 tx=0x7fa5b40045e0 comp rx=0 tx=0).stop 2024-09-19T06:16:10.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.920+0000 7fa5c9a29640 1 -- 172.21.15.53:0/1040375610 shutdown_connections 2024-09-19T06:16:10.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.920+0000 7fa5c9a29640 1 --2- 172.21.15.53:0/1040375610 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fa5ac06e2b0 0x7fa5ac070770 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:10.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.920+0000 7fa5c9a29640 1 --2- 172.21.15.53:0/1040375610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5c4107900 0x7fa5c419dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:10.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.920+0000 7fa5c9a29640 1 --2- 172.21.15.53:0/1040375610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5c4101900 0x7fa5c419d670 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:10.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.920+0000 7fa5c9a29640 1 -- 172.21.15.53:0/1040375610 >> 172.21.15.53:0/1040375610 conn(0x7fa5c40fd600 msgr2=0x7fa5c40fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:10.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.920+0000 7fa5c9a29640 1 -- 172.21.15.53:0/1040375610 shutdown_connections 2024-09-19T06:16:10.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:09.920+0000 7fa5c9a29640 1 -- 172.21.15.53:0/1040375610 wait complete. 2024-09-19T06:16:10.478 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:10.911 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378655 got 81604378656 for osd.2 2024-09-19T06:16:10.911 DEBUG:teuthology.parallel:result is None 2024-09-19T06:16:11.129 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:10 smithi053 ceph-mon[27968]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:11.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:10 smithi155 ceph-mon[35292]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:11.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.283+0000 7f45f0a49640 1 -- 172.21.15.53:0/4205239207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45ec107900 msgr2=0x7f45ec107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:11.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.283+0000 7f45f0a49640 1 --2- 172.21.15.53:0/4205239207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45ec107900 0x7f45ec107ce0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f45dc009880 tx=0x7f45dc02a4a0 comp rx=0 tx=0).stop 2024-09-19T06:16:11.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.283+0000 7f45f0a49640 1 -- 172.21.15.53:0/4205239207 shutdown_connections 2024-09-19T06:16:11.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.283+0000 7f45f0a49640 1 --2- 172.21.15.53:0/4205239207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45ec101900 0x7f45ec101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:11.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.283+0000 7f45f0a49640 1 --2- 172.21.15.53:0/4205239207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45ec107900 0x7f45ec107ce0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:11.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.283+0000 7f45f0a49640 1 -- 172.21.15.53:0/4205239207 >> 172.21.15.53:0/4205239207 conn(0x7f45ec0fd600 msgr2=0x7f45ec0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:11.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.284+0000 7f45f0a49640 1 -- 172.21.15.53:0/4205239207 shutdown_connections 2024-09-19T06:16:11.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.284+0000 7f45f0a49640 1 -- 172.21.15.53:0/4205239207 wait complete. 2024-09-19T06:16:11.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.284+0000 7f45f0a49640 1 Processor -- start 2024-09-19T06:16:11.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.284+0000 7f45f0a49640 1 -- start start 2024-09-19T06:16:11.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.285+0000 7f45f0a49640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45ec101900 0x7f45ec19d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:11.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.285+0000 7f45f0a49640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45ec107900 0x7f45ec19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:11.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.285+0000 7f45f0a49640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec10ed20 con 0x7f45ec107900 2024-09-19T06:16:11.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.285+0000 7f45f0a49640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec10eba0 con 0x7f45ec101900 2024-09-19T06:16:11.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.285+0000 7f45e9d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45ec107900 0x7f45ec19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:11.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.285+0000 7f45e9d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45ec107900 0x7f45ec19db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41510/0 (socket says 172.21.15.53:41510) 2024-09-19T06:16:11.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.285+0000 7f45e9d9b640 1 -- 172.21.15.53:0/1290127611 learned_addr learned my addr 172.21.15.53:0/1290127611 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:11.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.285+0000 7f45ea59c640 1 --2- 172.21.15.53:0/1290127611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45ec101900 0x7f45ec19d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:11.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.285+0000 7f45e9d9b640 1 -- 172.21.15.53:0/1290127611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45ec101900 msgr2=0x7f45ec19d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:11.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.285+0000 7f45e9d9b640 1 --2- 172.21.15.53:0/1290127611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45ec101900 0x7f45ec19d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:11.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.285+0000 7f45e9d9b640 1 -- 172.21.15.53:0/1290127611 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45ec197700 con 0x7f45ec107900 2024-09-19T06:16:11.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.285+0000 7f45ea59c640 1 --2- 172.21.15.53:0/1290127611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45ec101900 0x7f45ec19d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:16:11.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.285+0000 7f45e9d9b640 1 --2- 172.21.15.53:0/1290127611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45ec107900 0x7f45ec19db70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f45e000d8d0 tx=0x7f45e000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:11.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.285+0000 7f45db7fe640 1 -- 172.21.15.53:0/1290127611 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f45e0017070 con 0x7f45ec107900 2024-09-19T06:16:11.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.285+0000 7f45db7fe640 1 -- 172.21.15.53:0/1290127611 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45e0010460 con 0x7f45ec107900 2024-09-19T06:16:11.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.285+0000 7f45db7fe640 1 -- 172.21.15.53:0/1290127611 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f45e000f5f0 con 0x7f45ec107900 2024-09-19T06:16:11.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.285+0000 7f45f0a49640 1 -- 172.21.15.53:0/1290127611 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45ec1979f0 con 0x7f45ec107900 2024-09-19T06:16:11.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.286+0000 7f45f0a49640 1 -- 172.21.15.53:0/1290127611 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45ec197f80 con 0x7f45ec107900 2024-09-19T06:16:11.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.287+0000 7f45f0a49640 1 -- 172.21.15.53:0/1290127611 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45ec103040 con 0x7f45ec107900 2024-09-19T06:16:11.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.287+0000 7f45db7fe640 1 -- 172.21.15.53:0/1290127611 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f45e0010610 con 0x7f45ec107900 2024-09-19T06:16:11.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.287+0000 7f45db7fe640 1 --2- 172.21.15.53:0/1290127611 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f45d406e2b0 0x7f45d4070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:11.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.288+0000 7f45db7fe640 1 -- 172.21.15.53:0/1290127611 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5502+0+0 (secure 0 0 0) 0x7f45e008e0b0 con 0x7f45ec107900 2024-09-19T06:16:11.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.288+0000 7f45ea59c640 1 --2- 172.21.15.53:0/1290127611 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f45d406e2b0 0x7f45d4070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:11.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.288+0000 7f45ea59c640 1 --2- 172.21.15.53:0/1290127611 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f45d406e2b0 0x7f45d4070770 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f45dc005ab0 tx=0x7f45dc005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:11.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.292+0000 7f45db7fe640 1 -- 172.21.15.53:0/1290127611 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f45e005b960 con 0x7f45ec107900 2024-09-19T06:16:11.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.473+0000 7f45f0a49640 1 -- 172.21.15.53:0/1290127611 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f45ec198c30 con 0x7f45ec107900 2024-09-19T06:16:11.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.473+0000 7f45db7fe640 1 -- 172.21.15.53:0/1290127611 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f45e005f610 con 0x7f45ec107900 2024-09-19T06:16:11.475 INFO:teuthology.orchestra.run.smithi053.stdout:133143986199 2024-09-19T06:16:11.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.476+0000 7f45f0a49640 1 -- 172.21.15.53:0/1290127611 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f45d406e2b0 msgr2=0x7f45d4070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:11.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.476+0000 7f45f0a49640 1 --2- 172.21.15.53:0/1290127611 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f45d406e2b0 0x7f45d4070770 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f45dc005ab0 tx=0x7f45dc005a20 comp rx=0 tx=0).stop 2024-09-19T06:16:11.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.476+0000 7f45f0a49640 1 -- 172.21.15.53:0/1290127611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45ec107900 msgr2=0x7f45ec19db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:11.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.476+0000 7f45f0a49640 1 --2- 172.21.15.53:0/1290127611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45ec107900 0x7f45ec19db70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f45e000d8d0 tx=0x7f45e000dd90 comp rx=0 tx=0).stop 2024-09-19T06:16:11.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.476+0000 7f45f0a49640 1 -- 172.21.15.53:0/1290127611 shutdown_connections 2024-09-19T06:16:11.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.476+0000 7f45f0a49640 1 --2- 172.21.15.53:0/1290127611 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f45d406e2b0 0x7f45d4070770 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:11.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.476+0000 7f45f0a49640 1 --2- 172.21.15.53:0/1290127611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45ec107900 0x7f45ec19db70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:11.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.476+0000 7f45f0a49640 1 --2- 172.21.15.53:0/1290127611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45ec101900 0x7f45ec19d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:11.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.476+0000 7f45f0a49640 1 -- 172.21.15.53:0/1290127611 >> 172.21.15.53:0/1290127611 conn(0x7f45ec0fd600 msgr2=0x7f45ec0ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:11.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.476+0000 7f45f0a49640 1 -- 172.21.15.53:0/1290127611 shutdown_connections 2024-09-19T06:16:11.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.477+0000 7f45f0a49640 1 -- 172.21.15.53:0/1290127611 wait complete. 2024-09-19T06:16:11.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.461+0000 7f9a54247640 1 -- 172.21.15.53:0/2639890990 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a4c102ab0 msgr2=0x7f9a4c102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:11.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.461+0000 7f9a54247640 1 --2- 172.21.15.53:0/2639890990 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a4c102ab0 0x7f9a4c102e90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9a48009880 tx=0x7f9a4802a530 comp rx=0 tx=0).stop 2024-09-19T06:16:11.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.461+0000 7f9a54247640 1 -- 172.21.15.53:0/2639890990 shutdown_connections 2024-09-19T06:16:11.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.461+0000 7f9a54247640 1 --2- 172.21.15.53:0/2639890990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a4c1033d0 0x7f9a4c10b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:11.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.461+0000 7f9a54247640 1 --2- 172.21.15.53:0/2639890990 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a4c102ab0 0x7f9a4c102e90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:11.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.461+0000 7f9a54247640 1 -- 172.21.15.53:0/2639890990 >> 172.21.15.53:0/2639890990 conn(0x7f9a4c0fc550 msgr2=0x7f9a4c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:11.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.461+0000 7f9a54247640 1 -- 172.21.15.53:0/2639890990 shutdown_connections 2024-09-19T06:16:11.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.461+0000 7f9a54247640 1 -- 172.21.15.53:0/2639890990 wait complete. 2024-09-19T06:16:11.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.461+0000 7f9a54247640 1 Processor -- start 2024-09-19T06:16:11.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.461+0000 7f9a54247640 1 -- start start 2024-09-19T06:16:11.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.462+0000 7f9a54247640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a4c102ab0 0x7f9a4c19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:11.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.462+0000 7f9a54247640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a4c1033d0 0x7f9a4c19dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:11.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.462+0000 7f9a54247640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c10ed50 con 0x7f9a4c102ab0 2024-09-19T06:16:11.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.462+0000 7f9a54247640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c10ebd0 con 0x7f9a4c1033d0 2024-09-19T06:16:11.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.462+0000 7f9a51fe3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a4c102ab0 0x7f9a4c19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:11.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.462+0000 7f9a51fe3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a4c102ab0 0x7f9a4c19d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34246/0 (socket says 172.21.15.53:34246) 2024-09-19T06:16:11.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.462+0000 7f9a51fe3640 1 -- 172.21.15.53:0/3178792770 learned_addr learned my addr 172.21.15.53:0/3178792770 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:11.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.462+0000 7f9a517e2640 1 --2- 172.21.15.53:0/3178792770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a4c1033d0 0x7f9a4c19dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:11.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.462+0000 7f9a51fe3640 1 -- 172.21.15.53:0/3178792770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a4c1033d0 msgr2=0x7f9a4c19dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:11.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.462+0000 7f9a51fe3640 1 --2- 172.21.15.53:0/3178792770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a4c1033d0 0x7f9a4c19dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:11.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.462+0000 7f9a51fe3640 1 -- 172.21.15.53:0/3178792770 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a4c197770 con 0x7f9a4c102ab0 2024-09-19T06:16:11.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.462+0000 7f9a51fe3640 1 --2- 172.21.15.53:0/3178792770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a4c102ab0 0x7f9a4c19d6a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9a4802aa50 tx=0x7f9a480045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:11.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.462+0000 7f9a3effd640 1 -- 172.21.15.53:0/3178792770 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9a4803f070 con 0x7f9a4c102ab0 2024-09-19T06:16:11.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.462+0000 7f9a3effd640 1 -- 172.21.15.53:0/3178792770 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a48004a50 con 0x7f9a4c102ab0 2024-09-19T06:16:11.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.463+0000 7f9a3effd640 1 -- 172.21.15.53:0/3178792770 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9a48035030 con 0x7f9a4c102ab0 2024-09-19T06:16:11.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.463+0000 7f9a54247640 1 -- 172.21.15.53:0/3178792770 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a4c197a00 con 0x7f9a4c102ab0 2024-09-19T06:16:11.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.463+0000 7f9a54247640 1 -- 172.21.15.53:0/3178792770 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a4c197f30 con 0x7f9a4c102ab0 2024-09-19T06:16:11.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.464+0000 7f9a54247640 1 -- 172.21.15.53:0/3178792770 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a4c06a580 con 0x7f9a4c102ab0 2024-09-19T06:16:11.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.464+0000 7f9a3effd640 1 -- 172.21.15.53:0/3178792770 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f9a48042070 con 0x7f9a4c102ab0 2024-09-19T06:16:11.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.465+0000 7f9a3effd640 1 --2- 172.21.15.53:0/3178792770 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f9a3806e2b0 0x7f9a38070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:11.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.465+0000 7f9a3effd640 1 -- 172.21.15.53:0/3178792770 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5502+0+0 (secure 0 0 0) 0x7f9a480aec60 con 0x7f9a4c102ab0 2024-09-19T06:16:11.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.465+0000 7f9a517e2640 1 --2- 172.21.15.53:0/3178792770 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f9a3806e2b0 0x7f9a38070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:11.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.466+0000 7f9a517e2640 1 --2- 172.21.15.53:0/3178792770 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f9a3806e2b0 0x7f9a38070770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c198fc0 tx=0x7f9a40005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:11.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.469+0000 7f9a3effd640 1 -- 172.21.15.53:0/3178792770 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9a4807c510 con 0x7f9a4c102ab0 2024-09-19T06:16:11.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.659+0000 7f9a54247640 1 -- 172.21.15.53:0/3178792770 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f9a4c198ca0 con 0x7f9a4c102ab0 2024-09-19T06:16:11.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.659+0000 7f9a3effd640 1 -- 172.21.15.53:0/3178792770 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9a480801c0 con 0x7f9a4c102ab0 2024-09-19T06:16:11.661 INFO:teuthology.orchestra.run.smithi053.stdout:201863462924 2024-09-19T06:16:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.661+0000 7f9a54247640 1 -- 172.21.15.53:0/3178792770 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f9a3806e2b0 msgr2=0x7f9a38070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.662+0000 7f9a54247640 1 --2- 172.21.15.53:0/3178792770 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f9a3806e2b0 0x7f9a38070770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c198fc0 tx=0x7f9a40005f90 comp rx=0 tx=0).stop 2024-09-19T06:16:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.662+0000 7f9a54247640 1 -- 172.21.15.53:0/3178792770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a4c102ab0 msgr2=0x7f9a4c19d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.662+0000 7f9a54247640 1 --2- 172.21.15.53:0/3178792770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a4c102ab0 0x7f9a4c19d6a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9a4802aa50 tx=0x7f9a480045e0 comp rx=0 tx=0).stop 2024-09-19T06:16:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.662+0000 7f9a54247640 1 -- 172.21.15.53:0/3178792770 shutdown_connections 2024-09-19T06:16:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.662+0000 7f9a54247640 1 --2- 172.21.15.53:0/3178792770 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f9a3806e2b0 0x7f9a38070770 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.662+0000 7f9a54247640 1 --2- 172.21.15.53:0/3178792770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a4c1033d0 0x7f9a4c19dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.662+0000 7f9a54247640 1 --2- 172.21.15.53:0/3178792770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a4c102ab0 0x7f9a4c19d6a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.662+0000 7f9a54247640 1 -- 172.21.15.53:0/3178792770 >> 172.21.15.53:0/3178792770 conn(0x7f9a4c0fc550 msgr2=0x7f9a4c107dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.662+0000 7f9a54247640 1 -- 172.21.15.53:0/3178792770 shutdown_connections 2024-09-19T06:16:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:11.662+0000 7f9a54247640 1 -- 172.21.15.53:0/3178792770 wait complete. 2024-09-19T06:16:12.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:11 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/1290127611' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T06:16:12.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:11 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/3178792770' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T06:16:12.317 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:11 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/1290127611' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T06:16:12.317 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:11 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/3178792770' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T06:16:12.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.400+0000 7f142378e640 1 -- 172.21.15.53:0/677577849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f141c102ab0 msgr2=0x7f141c102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:12.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.400+0000 7f142378e640 1 --2- 172.21.15.53:0/677577849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f141c102ab0 0x7f141c102e90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1418009880 tx=0x7f141802a530 comp rx=0 tx=0).stop 2024-09-19T06:16:12.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.400+0000 7f142378e640 1 -- 172.21.15.53:0/677577849 shutdown_connections 2024-09-19T06:16:12.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.400+0000 7f142378e640 1 --2- 172.21.15.53:0/677577849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f141c1033d0 0x7f141c10b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:12.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.400+0000 7f142378e640 1 --2- 172.21.15.53:0/677577849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f141c102ab0 0x7f141c102e90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:12.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.400+0000 7f142378e640 1 -- 172.21.15.53:0/677577849 >> 172.21.15.53:0/677577849 conn(0x7f141c0fc550 msgr2=0x7f141c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:12.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.400+0000 7f142378e640 1 -- 172.21.15.53:0/677577849 shutdown_connections 2024-09-19T06:16:12.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.400+0000 7f142378e640 1 -- 172.21.15.53:0/677577849 wait complete. 2024-09-19T06:16:12.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.401+0000 7f142378e640 1 Processor -- start 2024-09-19T06:16:12.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.401+0000 7f142378e640 1 -- start start 2024-09-19T06:16:12.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.401+0000 7f142378e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f141c102ab0 0x7f141c19d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:12.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.401+0000 7f142378e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f141c1033d0 0x7f141c19db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:12.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.401+0000 7f142378e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f141c10ed30 con 0x7f141c1033d0 2024-09-19T06:16:12.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.401+0000 7f142378e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f141c10ebb0 con 0x7f141c102ab0 2024-09-19T06:16:12.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.402+0000 7f1420d29640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f141c1033d0 0x7f141c19db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:12.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.402+0000 7f1420d29640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f141c1033d0 0x7f141c19db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34290/0 (socket says 172.21.15.53:34290) 2024-09-19T06:16:12.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.402+0000 7f1420d29640 1 -- 172.21.15.53:0/2470158638 learned_addr learned my addr 172.21.15.53:0/2470158638 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:12.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.402+0000 7f142152a640 1 --2- 172.21.15.53:0/2470158638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f141c102ab0 0x7f141c19d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:12.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.402+0000 7f1420d29640 1 -- 172.21.15.53:0/2470158638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f141c102ab0 msgr2=0x7f141c19d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:12.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.402+0000 7f1420d29640 1 --2- 172.21.15.53:0/2470158638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f141c102ab0 0x7f141c19d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:12.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.402+0000 7f1420d29640 1 -- 172.21.15.53:0/2470158638 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141c1976f0 con 0x7f141c1033d0 2024-09-19T06:16:12.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.402+0000 7f1420d29640 1 --2- 172.21.15.53:0/2470158638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f141c1033d0 0x7f141c19db60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f140c00e970 tx=0x7f140c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:12.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.402+0000 7f14127fc640 1 -- 172.21.15.53:0/2470158638 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f140c009590 con 0x7f141c1033d0 2024-09-19T06:16:12.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.402+0000 7f142378e640 1 -- 172.21.15.53:0/2470158638 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141c1979e0 con 0x7f141c1033d0 2024-09-19T06:16:12.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.402+0000 7f14127fc640 1 -- 172.21.15.53:0/2470158638 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f140c010430 con 0x7f141c1033d0 2024-09-19T06:16:12.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.402+0000 7f14127fc640 1 -- 172.21.15.53:0/2470158638 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f140c018650 con 0x7f141c1033d0 2024-09-19T06:16:12.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.403+0000 7f142378e640 1 -- 172.21.15.53:0/2470158638 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f141c197f70 con 0x7f141c1033d0 2024-09-19T06:16:12.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.404+0000 7f142378e640 1 -- 172.21.15.53:0/2470158638 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f141c06a580 con 0x7f141c1033d0 2024-09-19T06:16:12.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.405+0000 7f14127fc640 1 -- 172.21.15.53:0/2470158638 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f140c0105e0 con 0x7f141c1033d0 2024-09-19T06:16:12.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.405+0000 7f14127fc640 1 --2- 172.21.15.53:0/2470158638 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f140806e2b0 0x7f1408070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:12.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.405+0000 7f14127fc640 1 -- 172.21.15.53:0/2470158638 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5502+0+0 (secure 0 0 0) 0x7f140c08e850 con 0x7f141c1033d0 2024-09-19T06:16:12.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.406+0000 7f142152a640 1 --2- 172.21.15.53:0/2470158638 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f140806e2b0 0x7f1408070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:12.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.406+0000 7f142152a640 1 --2- 172.21.15.53:0/2470158638 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f140806e2b0 0x7f1408070770 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1418005ab0 tx=0x7f1418005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:12.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.410+0000 7f14127fc640 1 -- 172.21.15.53:0/2470158638 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f140c05c100 con 0x7f141c1033d0 2024-09-19T06:16:12.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.592+0000 7f142378e640 1 -- 172.21.15.53:0/2470158638 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f141c105f80 con 0x7f141c1033d0 2024-09-19T06:16:12.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.592+0000 7f14127fc640 1 -- 172.21.15.53:0/2470158638 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f140c05fdb0 con 0x7f141c1033d0 2024-09-19T06:16:12.676 INFO:teuthology.orchestra.run.smithi053.stdout:240518168584 2024-09-19T06:16:12.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.594+0000 7f142378e640 1 -- 172.21.15.53:0/2470158638 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f140806e2b0 msgr2=0x7f1408070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:12.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.595+0000 7f142378e640 1 --2- 172.21.15.53:0/2470158638 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f140806e2b0 0x7f1408070770 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1418005ab0 tx=0x7f1418005a20 comp rx=0 tx=0).stop 2024-09-19T06:16:12.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.595+0000 7f142378e640 1 -- 172.21.15.53:0/2470158638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f141c1033d0 msgr2=0x7f141c19db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:12.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.595+0000 7f142378e640 1 --2- 172.21.15.53:0/2470158638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f141c1033d0 0x7f141c19db60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f140c00e970 tx=0x7f140c00ee30 comp rx=0 tx=0).stop 2024-09-19T06:16:12.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.595+0000 7f142378e640 1 -- 172.21.15.53:0/2470158638 shutdown_connections 2024-09-19T06:16:12.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.595+0000 7f142378e640 1 --2- 172.21.15.53:0/2470158638 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f140806e2b0 0x7f1408070770 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:12.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.595+0000 7f142378e640 1 --2- 172.21.15.53:0/2470158638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f141c1033d0 0x7f141c19db60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:12.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.595+0000 7f142378e640 1 --2- 172.21.15.53:0/2470158638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f141c102ab0 0x7f141c19d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:12.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.595+0000 7f142378e640 1 -- 172.21.15.53:0/2470158638 >> 172.21.15.53:0/2470158638 conn(0x7f141c0fc550 msgr2=0x7f141c108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:12.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.595+0000 7f142378e640 1 -- 172.21.15.53:0/2470158638 shutdown_connections 2024-09-19T06:16:12.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.595+0000 7f142378e640 1 -- 172.21.15.53:0/2470158638 wait complete. 2024-09-19T06:16:13.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.994+0000 7fcd7cce1640 1 -- 172.21.15.53:0/3407183744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd781067f0 msgr2=0x7fcd78106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:13.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.994+0000 7fcd7cce1640 1 --2- 172.21.15.53:0/3407183744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd781067f0 0x7fcd78106bd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009880 tx=0x7fcd6802a4a0 comp rx=0 tx=0).stop 2024-09-19T06:16:13.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.994+0000 7fcd7cce1640 1 -- 172.21.15.53:0/3407183744 shutdown_connections 2024-09-19T06:16:13.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.994+0000 7fcd7cce1640 1 --2- 172.21.15.53:0/3407183744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd781007f0 0x7fcd78100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:13.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.994+0000 7fcd7cce1640 1 --2- 172.21.15.53:0/3407183744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd781067f0 0x7fcd78106bd0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:13.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.994+0000 7fcd7cce1640 1 -- 172.21.15.53:0/3407183744 >> 172.21.15.53:0/3407183744 conn(0x7fcd780fc550 msgr2=0x7fcd780fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:13.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.994+0000 7fcd7cce1640 1 -- 172.21.15.53:0/3407183744 shutdown_connections 2024-09-19T06:16:13.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.994+0000 7fcd7cce1640 1 -- 172.21.15.53:0/3407183744 wait complete. 2024-09-19T06:16:13.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.995+0000 7fcd7cce1640 1 Processor -- start 2024-09-19T06:16:13.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.995+0000 7fcd7cce1640 1 -- start start 2024-09-19T06:16:13.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.996+0000 7fcd7cce1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd781007f0 0x7fcd7819d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:13.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.996+0000 7fcd7cce1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd781067f0 0x7fcd7819db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:13.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.996+0000 7fcd7cce1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7810dc10 con 0x7fcd781067f0 2024-09-19T06:16:13.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.996+0000 7fcd7cce1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7810da90 con 0x7fcd781007f0 2024-09-19T06:16:13.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.996+0000 7fcd75d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd781067f0 0x7fcd7819db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:13.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.996+0000 7fcd75d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd781067f0 0x7fcd7819db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34294/0 (socket says 172.21.15.53:34294) 2024-09-19T06:16:13.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.996+0000 7fcd75d9b640 1 -- 172.21.15.53:0/229236130 learned_addr learned my addr 172.21.15.53:0/229236130 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:13.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.996+0000 7fcd7659c640 1 --2- 172.21.15.53:0/229236130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd781007f0 0x7fcd7819d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:13.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.996+0000 7fcd75d9b640 1 -- 172.21.15.53:0/229236130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd781007f0 msgr2=0x7fcd7819d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:13.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.996+0000 7fcd75d9b640 1 --2- 172.21.15.53:0/229236130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd781007f0 0x7fcd7819d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:13.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.996+0000 7fcd75d9b640 1 -- 172.21.15.53:0/229236130 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd781976d0 con 0x7fcd781067f0 2024-09-19T06:16:13.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.996+0000 7fcd7659c640 1 --2- 172.21.15.53:0/229236130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd781007f0 0x7fcd7819d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:16:13.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.996+0000 7fcd75d9b640 1 --2- 172.21.15.53:0/229236130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd781067f0 0x7fcd7819db40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00c8a0 tx=0x7fcd6c00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:13.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.996+0000 7fcd677fe640 1 -- 172.21.15.53:0/229236130 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcd6c0079b0 con 0x7fcd781067f0 2024-09-19T06:16:13.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.996+0000 7fcd7cce1640 1 -- 172.21.15.53:0/229236130 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd781979c0 con 0x7fcd781067f0 2024-09-19T06:16:13.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.996+0000 7fcd677fe640 1 -- 172.21.15.53:0/229236130 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd6c00f450 con 0x7fcd781067f0 2024-09-19T06:16:13.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.997+0000 7fcd677fe640 1 -- 172.21.15.53:0/229236130 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcd6c018630 con 0x7fcd781067f0 2024-09-19T06:16:13.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.997+0000 7fcd7cce1640 1 -- 172.21.15.53:0/229236130 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd78197f50 con 0x7fcd781067f0 2024-09-19T06:16:13.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.998+0000 7fcd7cce1640 1 -- 172.21.15.53:0/229236130 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd78101f30 con 0x7fcd781067f0 2024-09-19T06:16:13.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.998+0000 7fcd677fe640 1 -- 172.21.15.53:0/229236130 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7fcd6c007b50 con 0x7fcd781067f0 2024-09-19T06:16:13.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.999+0000 7fcd677fe640 1 --2- 172.21.15.53:0/229236130 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fcd6006e2b0 0x7fcd60070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:13.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.999+0000 7fcd677fe640 1 -- 172.21.15.53:0/229236130 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5502+0+0 (secure 0 0 0) 0x7fcd6c08e540 con 0x7fcd781067f0 2024-09-19T06:16:13.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.999+0000 7fcd7659c640 1 --2- 172.21.15.53:0/229236130 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fcd6006e2b0 0x7fcd60070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:13.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.999+0000 7fcd7659c640 1 --2- 172.21.15.53:0/229236130 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fcd6006e2b0 0x7fcd60070770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcd68006010 tx=0x7fcd680095f0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:13.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:13.003+0000 7fcd677fe640 1 -- 172.21.15.53:0/229236130 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcd6c014030 con 0x7fcd781067f0 2024-09-19T06:16:13.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.201+0000 7f006cec8640 1 -- 172.21.15.53:0/4066389383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0068101900 msgr2=0x7f0068101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:13.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.201+0000 7f006cec8640 1 --2- 172.21.15.53:0/4066389383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0068101900 0x7f0068101d60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f005c009880 tx=0x7f005c02a4a0 comp rx=0 tx=0).stop 2024-09-19T06:16:13.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.201+0000 7f006cec8640 1 -- 172.21.15.53:0/4066389383 shutdown_connections 2024-09-19T06:16:13.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.201+0000 7f006cec8640 1 --2- 172.21.15.53:0/4066389383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0068101900 0x7f0068101d60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:13.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.201+0000 7f006cec8640 1 --2- 172.21.15.53:0/4066389383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0068107900 0x7f0068107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:13.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.201+0000 7f006cec8640 1 -- 172.21.15.53:0/4066389383 >> 172.21.15.53:0/4066389383 conn(0x7f00680fd600 msgr2=0x7f00680ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:13.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.201+0000 7f006cec8640 1 -- 172.21.15.53:0/4066389383 shutdown_connections 2024-09-19T06:16:13.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.201+0000 7f006cec8640 1 -- 172.21.15.53:0/4066389383 wait complete. 2024-09-19T06:16:13.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.202+0000 7f006cec8640 1 Processor -- start 2024-09-19T06:16:13.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.202+0000 7f006cec8640 1 -- start start 2024-09-19T06:16:13.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.202+0000 7f006cec8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0068101900 0x7f006819d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:13.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.202+0000 7f006cec8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0068107900 0x7f006819dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:13.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.202+0000 7f006cec8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f006810ed40 con 0x7f0068101900 2024-09-19T06:16:13.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.202+0000 7f006cec8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f006810ebc0 con 0x7f0068107900 2024-09-19T06:16:13.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.202+0000 7f006659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0068101900 0x7f006819d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:13.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.203+0000 7f006659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0068101900 0x7f006819d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34270/0 (socket says 172.21.15.53:34270) 2024-09-19T06:16:13.125 INFO:teuthology.orchestra.run.smithi053.stdout:111669149723 2024-09-19T06:16:13.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.203+0000 7f006659c640 1 -- 172.21.15.53:0/1337753712 learned_addr learned my addr 172.21.15.53:0/1337753712 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:13.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.203+0000 7f0065d9b640 1 --2- 172.21.15.53:0/1337753712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0068107900 0x7f006819dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:13.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.203+0000 7f006659c640 1 -- 172.21.15.53:0/1337753712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0068107900 msgr2=0x7f006819dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:13.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.203+0000 7f006659c640 1 --2- 172.21.15.53:0/1337753712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0068107900 0x7f006819dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:13.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.203+0000 7f006659c640 1 -- 172.21.15.53:0/1337753712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0068197740 con 0x7f0068101900 2024-09-19T06:16:13.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.203+0000 7f006659c640 1 --2- 172.21.15.53:0/1337753712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0068101900 0x7f006819d670 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f005800d6f0 tx=0x7f005800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:13.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.203+0000 7f00577fe640 1 -- 172.21.15.53:0/1337753712 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0058017070 con 0x7f0068101900 2024-09-19T06:16:13.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.203+0000 7f00577fe640 1 -- 172.21.15.53:0/1337753712 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f005800de00 con 0x7f0068101900 2024-09-19T06:16:13.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.203+0000 7f006cec8640 1 -- 172.21.15.53:0/1337753712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0068197a30 con 0x7f0068101900 2024-09-19T06:16:13.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.203+0000 7f00577fe640 1 -- 172.21.15.53:0/1337753712 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0058004aa0 con 0x7f0068101900 2024-09-19T06:16:13.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.203+0000 7f006cec8640 1 -- 172.21.15.53:0/1337753712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0068197fc0 con 0x7f0068101900 2024-09-19T06:16:13.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.204+0000 7f006cec8640 1 -- 172.21.15.53:0/1337753712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0068103040 con 0x7f0068101900 2024-09-19T06:16:13.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.205+0000 7f00577fe640 1 -- 172.21.15.53:0/1337753712 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f0058009a30 con 0x7f0068101900 2024-09-19T06:16:13.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.205+0000 7f00577fe640 1 --2- 172.21.15.53:0/1337753712 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f005006e310 0x7f00500707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:13.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.205+0000 7f00577fe640 1 -- 172.21.15.53:0/1337753712 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5502+0+0 (secure 0 0 0) 0x7f005808dd90 con 0x7f0068101900 2024-09-19T06:16:13.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.205+0000 7f0065d9b640 1 --2- 172.21.15.53:0/1337753712 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f005006e310 0x7f00500707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:13.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.206+0000 7f0065d9b640 1 --2- 172.21.15.53:0/1337753712 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f005006e310 0x7f00500707d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0068198f90 tx=0x7f005c005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:13.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.210+0000 7f00577fe640 1 -- 172.21.15.53:0/1337753712 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f005805b6c0 con 0x7f0068101900 2024-09-19T06:16:13.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.388+0000 7f006cec8640 1 -- 172.21.15.53:0/1337753712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f0068109e90 con 0x7f0068101900 2024-09-19T06:16:13.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.389+0000 7f00577fe640 1 -- 172.21.15.53:0/1337753712 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f005805f370 con 0x7f0068101900 2024-09-19T06:16:13.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.391+0000 7f006cec8640 1 -- 172.21.15.53:0/1337753712 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f005006e310 msgr2=0x7f00500707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:13.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.391+0000 7f006cec8640 1 --2- 172.21.15.53:0/1337753712 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f005006e310 0x7f00500707d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0068198f90 tx=0x7f005c005a20 comp rx=0 tx=0).stop 2024-09-19T06:16:13.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.391+0000 7f006cec8640 1 -- 172.21.15.53:0/1337753712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0068101900 msgr2=0x7f006819d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:13.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.391+0000 7f006cec8640 1 --2- 172.21.15.53:0/1337753712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0068101900 0x7f006819d670 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f005800d6f0 tx=0x7f005800dbb0 comp rx=0 tx=0).stop 2024-09-19T06:16:13.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.391+0000 7f006cec8640 1 -- 172.21.15.53:0/1337753712 shutdown_connections 2024-09-19T06:16:13.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.391+0000 7f006cec8640 1 --2- 172.21.15.53:0/1337753712 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f005006e310 0x7f00500707d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:13.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.391+0000 7f006cec8640 1 --2- 172.21.15.53:0/1337753712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0068107900 0x7f006819dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:13.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.391+0000 7f006cec8640 1 --2- 172.21.15.53:0/1337753712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0068101900 0x7f006819d670 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:13.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.391+0000 7f006cec8640 1 -- 172.21.15.53:0/1337753712 >> 172.21.15.53:0/1337753712 conn(0x7f00680fd600 msgr2=0x7f00680fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:13.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.391+0000 7f006cec8640 1 -- 172.21.15.53:0/1337753712 shutdown_connections 2024-09-19T06:16:13.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:12.391+0000 7f006cec8640 1 -- 172.21.15.53:0/1337753712 wait complete. 2024-09-19T06:16:13.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:13.181+0000 7fcd7cce1640 1 -- 172.21.15.53:0/229236130 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fcd78198c00 con 0x7fcd781067f0 2024-09-19T06:16:13.183 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:12 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/1337753712' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T06:16:13.183 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:12 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/2470158638' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T06:16:13.183 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:12 smithi053 ceph-mon[27968]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:13.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:13.181+0000 7fcd677fe640 1 -- 172.21.15.53:0/229236130 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcd6c05bdf0 con 0x7fcd781067f0 2024-09-19T06:16:13.183 INFO:teuthology.orchestra.run.smithi053.stdout:167503724562 2024-09-19T06:16:13.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:13.183+0000 7fcd7cce1640 1 -- 172.21.15.53:0/229236130 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fcd6006e2b0 msgr2=0x7fcd60070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:13.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:13.183+0000 7fcd7cce1640 1 --2- 172.21.15.53:0/229236130 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fcd6006e2b0 0x7fcd60070770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcd68006010 tx=0x7fcd680095f0 comp rx=0 tx=0).stop 2024-09-19T06:16:13.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:13.183+0000 7fcd7cce1640 1 -- 172.21.15.53:0/229236130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd781067f0 msgr2=0x7fcd7819db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:13.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:13.183+0000 7fcd7cce1640 1 --2- 172.21.15.53:0/229236130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd781067f0 0x7fcd7819db40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00c8a0 tx=0x7fcd6c00cd60 comp rx=0 tx=0).stop 2024-09-19T06:16:13.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:13.183+0000 7fcd7cce1640 1 -- 172.21.15.53:0/229236130 shutdown_connections 2024-09-19T06:16:13.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:13.183+0000 7fcd7cce1640 1 --2- 172.21.15.53:0/229236130 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fcd6006e2b0 0x7fcd60070770 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:13.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:13.183+0000 7fcd7cce1640 1 --2- 172.21.15.53:0/229236130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd781067f0 0x7fcd7819db40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:13.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:13.183+0000 7fcd7cce1640 1 --2- 172.21.15.53:0/229236130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd781007f0 0x7fcd7819d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:13.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:13.183+0000 7fcd7cce1640 1 -- 172.21.15.53:0/229236130 >> 172.21.15.53:0/229236130 conn(0x7fcd780fc550 msgr2=0x7fcd78069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:13.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:13.183+0000 7fcd7cce1640 1 -- 172.21.15.53:0/229236130 shutdown_connections 2024-09-19T06:16:13.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:13.183+0000 7fcd7cce1640 1 -- 172.21.15.53:0/229236130 wait complete. 2024-09-19T06:16:13.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:12 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/1337753712' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T06:16:13.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:12 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/2470158638' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T06:16:13.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:12 smithi155 ceph-mon[35292]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:13.676 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149722 got 111669149723 for osd.3 2024-09-19T06:16:13.676 DEBUG:teuthology.parallel:result is None 2024-09-19T06:16:14.123 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168583 got 240518168584 for osd.7 2024-09-19T06:16:14.123 DEBUG:teuthology.parallel:result is None 2024-09-19T06:16:14.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:13 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/229236130' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T06:16:14.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:13 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:16:14.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:13 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/229236130' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T06:16:14.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:13 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:16:14.486 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462923 got 201863462924 for osd.6 2024-09-19T06:16:14.486 DEBUG:teuthology.parallel:result is None 2024-09-19T06:16:14.890 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986197 got 133143986199 for osd.4 2024-09-19T06:16:14.890 DEBUG:teuthology.parallel:result is None 2024-09-19T06:16:15.122 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:15.157 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:15.249 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:14 smithi053 ceph-mon[27968]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:15.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:14 smithi155 ceph-mon[35292]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:15.616 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724560 got 167503724562 for osd.5 2024-09-19T06:16:15.617 DEBUG:teuthology.parallel:result is None 2024-09-19T06:16:16.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.215+0000 7fd6e2c50640 1 -- 172.21.15.53:0/4208521128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6dc1067f0 msgr2=0x7fd6dc106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:16.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.215+0000 7fd6e2c50640 1 --2- 172.21.15.53:0/4208521128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6dc1067f0 0x7fd6dc106bd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0009880 tx=0x7fd6d002a4a0 comp rx=0 tx=0).stop 2024-09-19T06:16:16.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.215+0000 7fd6e2c50640 1 -- 172.21.15.53:0/4208521128 shutdown_connections 2024-09-19T06:16:16.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.216+0000 7fd6e2c50640 1 --2- 172.21.15.53:0/4208521128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6dc1007f0 0x7fd6dc100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:16.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.216+0000 7fd6e2c50640 1 --2- 172.21.15.53:0/4208521128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6dc1067f0 0x7fd6dc106bd0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:16.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.216+0000 7fd6e2c50640 1 -- 172.21.15.53:0/4208521128 >> 172.21.15.53:0/4208521128 conn(0x7fd6dc0fc550 msgr2=0x7fd6dc0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:16.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.216+0000 7fd6e2c50640 1 -- 172.21.15.53:0/4208521128 shutdown_connections 2024-09-19T06:16:16.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.216+0000 7fd6e2c50640 1 -- 172.21.15.53:0/4208521128 wait complete. 2024-09-19T06:16:16.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.216+0000 7fd6e2c50640 1 Processor -- start 2024-09-19T06:16:16.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.216+0000 7fd6e2c50640 1 -- start start 2024-09-19T06:16:16.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.217+0000 7fd6e2c50640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6dc1007f0 0x7fd6dc19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:16.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.217+0000 7fd6e2c50640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6dc1067f0 0x7fd6dc19fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:16.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.217+0000 7fd6e2c50640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc10dc90 con 0x7fd6dc1007f0 2024-09-19T06:16:16.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.217+0000 7fd6e2c50640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc10db10 con 0x7fd6dc1067f0 2024-09-19T06:16:16.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.217+0000 7fd6e09ec640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6dc1007f0 0x7fd6dc19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:16.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.217+0000 7fd6e09ec640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6dc1007f0 0x7fd6dc19f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34314/0 (socket says 172.21.15.53:34314) 2024-09-19T06:16:16.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.217+0000 7fd6e09ec640 1 -- 172.21.15.53:0/4004248237 learned_addr learned my addr 172.21.15.53:0/4004248237 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:16.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.217+0000 7fd6dbfff640 1 --2- 172.21.15.53:0/4004248237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6dc1067f0 0x7fd6dc19fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:16.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.217+0000 7fd6e09ec640 1 -- 172.21.15.53:0/4004248237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6dc1067f0 msgr2=0x7fd6dc19fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:16.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.217+0000 7fd6e09ec640 1 --2- 172.21.15.53:0/4004248237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6dc1067f0 0x7fd6dc19fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:16.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.217+0000 7fd6e09ec640 1 -- 172.21.15.53:0/4004248237 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6dc1a0420 con 0x7fd6dc1007f0 2024-09-19T06:16:16.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.217+0000 7fd6dbfff640 1 --2- 172.21.15.53:0/4004248237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6dc1067f0 0x7fd6dc19fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:16:16.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.217+0000 7fd6e09ec640 1 --2- 172.21.15.53:0/4004248237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6dc1007f0 0x7fd6dc19f830 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0006010 tx=0x7fd6d00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:16.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.217+0000 7fd6d9ffb640 1 -- 172.21.15.53:0/4004248237 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd6d003f070 con 0x7fd6dc1007f0 2024-09-19T06:16:16.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.217+0000 7fd6e2c50640 1 -- 172.21.15.53:0/4004248237 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6dc199ab0 con 0x7fd6dc1007f0 2024-09-19T06:16:16.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.217+0000 7fd6d9ffb640 1 -- 172.21.15.53:0/4004248237 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6d0003aa0 con 0x7fd6dc1007f0 2024-09-19T06:16:16.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.217+0000 7fd6d9ffb640 1 -- 172.21.15.53:0/4004248237 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd6d0035030 con 0x7fd6dc1007f0 2024-09-19T06:16:16.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.218+0000 7fd6e2c50640 1 -- 172.21.15.53:0/4004248237 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6dc199fb0 con 0x7fd6dc1007f0 2024-09-19T06:16:16.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.218+0000 7fd6e2c50640 1 -- 172.21.15.53:0/4004248237 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6dc101f30 con 0x7fd6dc1007f0 2024-09-19T06:16:16.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.220+0000 7fd6d9ffb640 1 -- 172.21.15.53:0/4004248237 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7fd6d0042070 con 0x7fd6dc1007f0 2024-09-19T06:16:16.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.220+0000 7fd6d9ffb640 1 --2- 172.21.15.53:0/4004248237 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fd6cc06e2b0 0x7fd6cc070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:16.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.220+0000 7fd6d9ffb640 1 -- 172.21.15.53:0/4004248237 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5502+0+0 (secure 0 0 0) 0x7fd6d00af400 con 0x7fd6dc1007f0 2024-09-19T06:16:16.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.221+0000 7fd6dbfff640 1 --2- 172.21.15.53:0/4004248237 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fd6cc06e2b0 0x7fd6cc070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:16.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.221+0000 7fd6dbfff640 1 --2- 172.21.15.53:0/4004248237 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fd6cc06e2b0 0x7fd6cc070770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8006fd0 tx=0x7fd6c8008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:16.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.224+0000 7fd6d9ffb640 1 -- 172.21.15.53:0/4004248237 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd6d007cd30 con 0x7fd6dc1007f0 2024-09-19T06:16:16.403 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.402+0000 7fd6e2c50640 1 -- 172.21.15.53:0/4004248237 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fd6dc19a320 con 0x7fd6dc1007f0 2024-09-19T06:16:16.403 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.402+0000 7fd6d9ffb640 1 -- 172.21.15.53:0/4004248237 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd6d00ade20 con 0x7fd6dc1007f0 2024-09-19T06:16:16.404 INFO:teuthology.orchestra.run.smithi053.stdout:60129542184 2024-09-19T06:16:16.406 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.405+0000 7fd6e2c50640 1 -- 172.21.15.53:0/4004248237 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fd6cc06e2b0 msgr2=0x7fd6cc070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:16.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.405+0000 7fd6e2c50640 1 --2- 172.21.15.53:0/4004248237 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fd6cc06e2b0 0x7fd6cc070770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8006fd0 tx=0x7fd6c8008040 comp rx=0 tx=0).stop 2024-09-19T06:16:16.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.405+0000 7fd6e2c50640 1 -- 172.21.15.53:0/4004248237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6dc1007f0 msgr2=0x7fd6dc19f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:16.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.405+0000 7fd6e2c50640 1 --2- 172.21.15.53:0/4004248237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6dc1007f0 0x7fd6dc19f830 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0006010 tx=0x7fd6d00048f0 comp rx=0 tx=0).stop 2024-09-19T06:16:16.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.405+0000 7fd6e2c50640 1 -- 172.21.15.53:0/4004248237 shutdown_connections 2024-09-19T06:16:16.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.405+0000 7fd6e2c50640 1 --2- 172.21.15.53:0/4004248237 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fd6cc06e2b0 0x7fd6cc070770 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:16.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.405+0000 7fd6e2c50640 1 --2- 172.21.15.53:0/4004248237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6dc1067f0 0x7fd6dc19fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:16.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.405+0000 7fd6e2c50640 1 --2- 172.21.15.53:0/4004248237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6dc1007f0 0x7fd6dc19f830 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:16.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.405+0000 7fd6e2c50640 1 -- 172.21.15.53:0/4004248237 >> 172.21.15.53:0/4004248237 conn(0x7fd6dc0fc550 msgr2=0x7fd6dc06a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:16.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.405+0000 7fd6e2c50640 1 -- 172.21.15.53:0/4004248237 shutdown_connections 2024-09-19T06:16:16.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.405+0000 7fd6e2c50640 1 -- 172.21.15.53:0/4004248237 wait complete. 2024-09-19T06:16:16.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.406+0000 7f5822469640 1 -- 172.21.15.53:0/1616006328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f581c103b00 msgr2=0x7f581c103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:16.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.406+0000 7f5822469640 1 --2- 172.21.15.53:0/1616006328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f581c103b00 0x7f581c103f60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5810009880 tx=0x7f581002a4a0 comp rx=0 tx=0).stop 2024-09-19T06:16:16.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.406+0000 7f5822469640 1 -- 172.21.15.53:0/1616006328 shutdown_connections 2024-09-19T06:16:16.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.406+0000 7f5822469640 1 --2- 172.21.15.53:0/1616006328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f581c103b00 0x7f581c103f60 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:16.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.406+0000 7f5822469640 1 --2- 172.21.15.53:0/1616006328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f581c109b90 0x7f581c109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:16.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.406+0000 7f5822469640 1 -- 172.21.15.53:0/1616006328 >> 172.21.15.53:0/1616006328 conn(0x7f581c0ff840 msgr2=0x7f581c101c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:16.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.406+0000 7f5822469640 1 -- 172.21.15.53:0/1616006328 shutdown_connections 2024-09-19T06:16:16.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.406+0000 7f5822469640 1 -- 172.21.15.53:0/1616006328 wait complete. 2024-09-19T06:16:16.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.407+0000 7f5822469640 1 Processor -- start 2024-09-19T06:16:16.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.407+0000 7f5822469640 1 -- start start 2024-09-19T06:16:16.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.408+0000 7f5822469640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f581c103b00 0x7f581c19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:16.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.408+0000 7f5822469640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f581c109b90 0x7f581c19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:16.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.408+0000 7f5822469640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f581c110f40 con 0x7f581c109b90 2024-09-19T06:16:16.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.408+0000 7f5822469640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f581c110dc0 con 0x7f581c103b00 2024-09-19T06:16:16.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.408+0000 7f581b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f581c109b90 0x7f581c19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:16.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.408+0000 7f581b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f581c109b90 0x7f581c19fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34326/0 (socket says 172.21.15.53:34326) 2024-09-19T06:16:16.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.408+0000 7f581b7fe640 1 -- 172.21.15.53:0/1744917193 learned_addr learned my addr 172.21.15.53:0/1744917193 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:16.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.408+0000 7f581bfff640 1 --2- 172.21.15.53:0/1744917193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f581c103b00 0x7f581c19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:16.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.408+0000 7f581b7fe640 1 -- 172.21.15.53:0/1744917193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f581c103b00 msgr2=0x7f581c19f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:16.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.408+0000 7f581b7fe640 1 --2- 172.21.15.53:0/1744917193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f581c103b00 0x7f581c19f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:16.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.408+0000 7f581b7fe640 1 -- 172.21.15.53:0/1744917193 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f581c1a0390 con 0x7f581c109b90 2024-09-19T06:16:16.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.408+0000 7f581b7fe640 1 --2- 172.21.15.53:0/1744917193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f581c109b90 0x7f581c19fce0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5810006010 tx=0x7f58100048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:16.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.409+0000 7f58197fa640 1 -- 172.21.15.53:0/1744917193 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f581003f070 con 0x7f581c109b90 2024-09-19T06:16:16.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.409+0000 7f58197fa640 1 -- 172.21.15.53:0/1744917193 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5810003aa0 con 0x7f581c109b90 2024-09-19T06:16:16.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.409+0000 7f5822469640 1 -- 172.21.15.53:0/1744917193 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f581c199a20 con 0x7f581c109b90 2024-09-19T06:16:16.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.409+0000 7f58197fa640 1 -- 172.21.15.53:0/1744917193 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5810035030 con 0x7f581c109b90 2024-09-19T06:16:16.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.409+0000 7f5822469640 1 -- 172.21.15.53:0/1744917193 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f581c199f20 con 0x7f581c109b90 2024-09-19T06:16:16.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.410+0000 7f5822469640 1 -- 172.21.15.53:0/1744917193 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f581c105200 con 0x7f581c109b90 2024-09-19T06:16:16.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.411+0000 7f58197fa640 1 -- 172.21.15.53:0/1744917193 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f5810042070 con 0x7f581c109b90 2024-09-19T06:16:16.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.411+0000 7f58197fa640 1 --2- 172.21.15.53:0/1744917193 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f580406e310 0x7f58040707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:16.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.412+0000 7f581bfff640 1 --2- 172.21.15.53:0/1744917193 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f580406e310 0x7f58040707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:16.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.412+0000 7f58197fa640 1 -- 172.21.15.53:0/1744917193 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5502+0+0 (secure 0 0 0) 0x7f58100aec10 con 0x7f581c109b90 2024-09-19T06:16:16.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.412+0000 7f581bfff640 1 --2- 172.21.15.53:0/1744917193 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f580406e310 0x7f58040707d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f580c0058c0 tx=0x7f580c00a250 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:16.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.417+0000 7f58197fa640 1 -- 172.21.15.53:0/1744917193 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f58100801f0 con 0x7f581c109b90 2024-09-19T06:16:16.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.609+0000 7f5822469640 1 -- 172.21.15.53:0/1744917193 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f581c19ae90 con 0x7f581c109b90 2024-09-19T06:16:16.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.610+0000 7f58197fa640 1 -- 172.21.15.53:0/1744917193 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5810084a20 con 0x7f581c109b90 2024-09-19T06:16:16.611 INFO:teuthology.orchestra.run.smithi053.stdout:38654705709 2024-09-19T06:16:16.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.613+0000 7f5822469640 1 -- 172.21.15.53:0/1744917193 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f580406e310 msgr2=0x7f58040707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:16.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.613+0000 7f5822469640 1 --2- 172.21.15.53:0/1744917193 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f580406e310 0x7f58040707d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f580c0058c0 tx=0x7f580c00a250 comp rx=0 tx=0).stop 2024-09-19T06:16:16.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.613+0000 7f5822469640 1 -- 172.21.15.53:0/1744917193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f581c109b90 msgr2=0x7f581c19fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:16.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.613+0000 7f5822469640 1 --2- 172.21.15.53:0/1744917193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f581c109b90 0x7f581c19fce0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5810006010 tx=0x7f58100048f0 comp rx=0 tx=0).stop 2024-09-19T06:16:16.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.613+0000 7f5822469640 1 -- 172.21.15.53:0/1744917193 shutdown_connections 2024-09-19T06:16:16.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.613+0000 7f5822469640 1 --2- 172.21.15.53:0/1744917193 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f580406e310 0x7f58040707d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:16.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.613+0000 7f5822469640 1 --2- 172.21.15.53:0/1744917193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f581c109b90 0x7f581c19fce0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:16.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.613+0000 7f5822469640 1 --2- 172.21.15.53:0/1744917193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f581c103b00 0x7f581c19f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:16.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.613+0000 7f5822469640 1 -- 172.21.15.53:0/1744917193 >> 172.21.15.53:0/1744917193 conn(0x7f581c0ff840 msgr2=0x7f581c100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:16.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.613+0000 7f5822469640 1 -- 172.21.15.53:0/1744917193 shutdown_connections 2024-09-19T06:16:16.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:16.613+0000 7f5822469640 1 -- 172.21.15.53:0/1744917193 wait complete. 2024-09-19T06:16:16.658 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:16 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/4004248237' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T06:16:16.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:16 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/4004248237' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T06:16:17.262 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542182 got 60129542184 for osd.1 2024-09-19T06:16:17.263 DEBUG:teuthology.parallel:result is None 2024-09-19T06:16:17.707 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:17 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/1744917193' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T06:16:17.708 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:17 smithi053 ceph-mon[27968]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:17.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:17 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/1744917193' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T06:16:17.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:17 smithi155 ceph-mon[35292]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:17.737 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705707 got 38654705709 for osd.0 2024-09-19T06:16:17.737 DEBUG:teuthology.parallel:result is None 2024-09-19T06:16:17.738 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T06:16:17.738 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T06:16:18.002 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:18.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.393+0000 7f86304bc640 1 -- 172.21.15.53:0/3343841415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8628102ac0 msgr2=0x7f8628102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:18.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.393+0000 7f86304bc640 1 --2- 172.21.15.53:0/3343841415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8628102ac0 0x7f8628102ea0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8624009c20 tx=0x7f862402aa00 comp rx=0 tx=0).stop 2024-09-19T06:16:18.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.393+0000 7f86304bc640 1 -- 172.21.15.53:0/3343841415 shutdown_connections 2024-09-19T06:16:18.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.393+0000 7f86304bc640 1 --2- 172.21.15.53:0/3343841415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86281033e0 0x7f862810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:18.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.393+0000 7f86304bc640 1 --2- 172.21.15.53:0/3343841415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8628102ac0 0x7f8628102ea0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:18.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.393+0000 7f86304bc640 1 -- 172.21.15.53:0/3343841415 >> 172.21.15.53:0/3343841415 conn(0x7f86280fc520 msgr2=0x7f86280fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:18.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.393+0000 7f86304bc640 1 -- 172.21.15.53:0/3343841415 shutdown_connections 2024-09-19T06:16:18.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.393+0000 7f86304bc640 1 -- 172.21.15.53:0/3343841415 wait complete. 2024-09-19T06:16:18.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.394+0000 7f86304bc640 1 Processor -- start 2024-09-19T06:16:18.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.394+0000 7f86304bc640 1 -- start start 2024-09-19T06:16:18.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.394+0000 7f86304bc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8628102ac0 0x7f862819d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:18.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.394+0000 7f86304bc640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86281033e0 0x7f862819db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:18.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.394+0000 7f86304bc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f862810ed40 con 0x7f8628102ac0 2024-09-19T06:16:18.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.394+0000 7f86304bc640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f862810ebc0 con 0x7f86281033e0 2024-09-19T06:16:18.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.395+0000 7f862e258640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8628102ac0 0x7f862819d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:18.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.395+0000 7f862e258640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8628102ac0 0x7f862819d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34338/0 (socket says 172.21.15.53:34338) 2024-09-19T06:16:18.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.395+0000 7f862e258640 1 -- 172.21.15.53:0/3614015156 learned_addr learned my addr 172.21.15.53:0/3614015156 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:18.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.395+0000 7f862da57640 1 --2- 172.21.15.53:0/3614015156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86281033e0 0x7f862819db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:18.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.395+0000 7f862e258640 1 -- 172.21.15.53:0/3614015156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86281033e0 msgr2=0x7f862819db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:18.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.395+0000 7f862e258640 1 --2- 172.21.15.53:0/3614015156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86281033e0 0x7f862819db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:18.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.395+0000 7f862e258640 1 -- 172.21.15.53:0/3614015156 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8628197720 con 0x7f8628102ac0 2024-09-19T06:16:18.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.395+0000 7f862da57640 1 --2- 172.21.15.53:0/3614015156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86281033e0 0x7f862819db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:16:18.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.395+0000 7f862e258640 1 --2- 172.21.15.53:0/3614015156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8628102ac0 0x7f862819d650 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f862402af20 tx=0x7f862402e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:18.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.395+0000 7f861f7fe640 1 -- 172.21.15.53:0/3614015156 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8624047070 con 0x7f8628102ac0 2024-09-19T06:16:18.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.395+0000 7f86304bc640 1 -- 172.21.15.53:0/3614015156 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86281979b0 con 0x7f8628102ac0 2024-09-19T06:16:18.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.395+0000 7f861f7fe640 1 -- 172.21.15.53:0/3614015156 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8624003e20 con 0x7f8628102ac0 2024-09-19T06:16:18.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.395+0000 7f861f7fe640 1 -- 172.21.15.53:0/3614015156 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8624033050 con 0x7f8628102ac0 2024-09-19T06:16:18.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.395+0000 7f86304bc640 1 -- 172.21.15.53:0/3614015156 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8628197ee0 con 0x7f8628102ac0 2024-09-19T06:16:18.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.396+0000 7f86304bc640 1 -- 172.21.15.53:0/3614015156 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f862806aa90 con 0x7f8628102ac0 2024-09-19T06:16:18.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.397+0000 7f861f7fe640 1 -- 172.21.15.53:0/3614015156 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f862404a070 con 0x7f8628102ac0 2024-09-19T06:16:18.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.397+0000 7f861f7fe640 1 --2- 172.21.15.53:0/3614015156 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f861406e2b0 0x7f8614070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:18.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.397+0000 7f861f7fe640 1 -- 172.21.15.53:0/3614015156 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5502+0+0 (secure 0 0 0) 0x7f862403e030 con 0x7f8628102ac0 2024-09-19T06:16:18.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.397+0000 7f862da57640 1 --2- 172.21.15.53:0/3614015156 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f861406e2b0 0x7f8614070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:18.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.398+0000 7f862da57640 1 --2- 172.21.15.53:0/3614015156 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f861406e2b0 0x7f8614070770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8628198f70 tx=0x7f8618005e30 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:18.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.402+0000 7f861f7fe640 1 -- 172.21.15.53:0/3614015156 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f862407cce0 con 0x7f8628102ac0 2024-09-19T06:16:18.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.574+0000 7f86304bc640 1 -- 172.21.15.53:0/3614015156 --> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8628198190 con 0x7f861406e2b0 2024-09-19T06:16:18.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.575+0000 7f861f7fe640 1 -- 172.21.15.53:0/3614015156 <== mgr.14217 v2:172.21.15.53:6800/3884355752 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+14526 (secure 0 0 0) 0x7f8628198190 con 0x7f861406e2b0 2024-09-19T06:16:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:16:18.576 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-19T06:16:18.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.577+0000 7f86304bc640 1 -- 172.21.15.53:0/3614015156 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f861406e2b0 msgr2=0x7f8614070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:18.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.577+0000 7f86304bc640 1 --2- 172.21.15.53:0/3614015156 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f861406e2b0 0x7f8614070770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8628198f70 tx=0x7f8618005e30 comp rx=0 tx=0).stop 2024-09-19T06:16:18.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.577+0000 7f86304bc640 1 -- 172.21.15.53:0/3614015156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8628102ac0 msgr2=0x7f862819d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:18.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.577+0000 7f86304bc640 1 --2- 172.21.15.53:0/3614015156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8628102ac0 0x7f862819d650 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f862402af20 tx=0x7f862402e720 comp rx=0 tx=0).stop 2024-09-19T06:16:18.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.577+0000 7f86304bc640 1 -- 172.21.15.53:0/3614015156 shutdown_connections 2024-09-19T06:16:18.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.577+0000 7f86304bc640 1 --2- 172.21.15.53:0/3614015156 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f861406e2b0 0x7f8614070770 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:18.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.577+0000 7f86304bc640 1 --2- 172.21.15.53:0/3614015156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86281033e0 0x7f862819db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:18.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.577+0000 7f86304bc640 1 --2- 172.21.15.53:0/3614015156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8628102ac0 0x7f862819d650 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:18.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.577+0000 7f86304bc640 1 -- 172.21.15.53:0/3614015156 >> 172.21.15.53:0/3614015156 conn(0x7f86280fc520 msgr2=0x7f8628103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:18.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.578+0000 7f86304bc640 1 -- 172.21.15.53:0/3614015156 shutdown_connections 2024-09-19T06:16:18.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:18.578+0000 7f86304bc640 1 -- 172.21.15.53:0/3614015156 wait complete. 2024-09-19T06:16:18.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:18 smithi155 ceph-mon[35292]: from='client.14470 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:16:18.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:18 smithi155 ceph-mon[35292]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:19.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:18 smithi053 ceph-mon[27968]: from='client.14470 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:16:19.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:18 smithi053 ceph-mon[27968]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:19.308 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2024-09-19T06:16:16.775241+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":749961216,"kb_used":239512,"kb_used_data":6604,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":749721704,"statfs":{"total":767960285184,"available":767715024896,"internally_reserved":0,"allocated":6762496,"data_stored":3783584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002506"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":214,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-19T06:15:51.825126+0000","last_change":"2024-09-19T06:15:51.824758+0000","last_active":"2024-09-19T06:15:51.825126+0000","last_peered":"2024-09-19T06:15:51.825126+0000","last_clean":"2024-09-19T06:15:51.825126+0000","last_became_active":"2024-09-19T06:15:51.824328+0000","last_became_peered":"2024-09-19T06:15:51.824328+0000","last_unstale":"2024-09-19T06:15:51.825126+0000","last_undegraded":"2024-09-19T06:15:51.825126+0000","last_fullsized":"2024-09-19T06:15:51.825126+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:13:39.618158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:13:39.618158+0000","last_clean_scrub_stamp":"2024-09-19T06:13:39.618158+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:53:49.590551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":56,"seq":240518168585,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28220,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716932,"statfs":{"total":95995035648,"available":95966138368,"internally_reserved":0,"allocated":1216512,"data_stored":841928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462925,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28224,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716928,"statfs":{"total":95995035648,"available":95966134272,"internally_reserved":0,"allocated":1216512,"data_stored":841928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724563,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27640,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717512,"statfs":{"total":95995035648,"available":95966732288,"internally_reserved":0,"allocated":622592,"data_stored":251560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986200,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27644,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717508,"statfs":{"total":95995035648,"available":95966728192,"internally_reserved":0,"allocated":622592,"data_stored":251560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149724,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":31796,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713356,"statfs":{"total":95995035648,"available":95962476544,"internally_reserved":0,"allocated":622592,"data_stored":251560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378658,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":31804,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713348,"statfs":{"total":95995035648,"available":95962468352,"internally_reserved":0,"allocated":622592,"data_stored":251560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542184,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":31800,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713352,"statfs":{"total":95995035648,"available":95962472448,"internally_reserved":0,"allocated":622592,"data_stored":251560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705710,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32384,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712768,"statfs":{"total":95995035648,"available":95961874432,"internally_reserved":0,"allocated":1216512,"data_stored":841928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T06:16:19.309 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T06:16:19.572 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:19.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.977+0000 7f5b98b31640 1 -- 172.21.15.53:0/2738661598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b941067f0 msgr2=0x7f5b94106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:19.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.977+0000 7f5b98b31640 1 --2- 172.21.15.53:0/2738661598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b941067f0 0x7f5b94106bd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5b84009880 tx=0x7f5b8402a4a0 comp rx=0 tx=0).stop 2024-09-19T06:16:19.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.977+0000 7f5b98b31640 1 -- 172.21.15.53:0/2738661598 shutdown_connections 2024-09-19T06:16:19.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.977+0000 7f5b98b31640 1 --2- 172.21.15.53:0/2738661598 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b941007f0 0x7f5b94100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:19.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.977+0000 7f5b98b31640 1 --2- 172.21.15.53:0/2738661598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b941067f0 0x7f5b94106bd0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:19.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.977+0000 7f5b98b31640 1 -- 172.21.15.53:0/2738661598 >> 172.21.15.53:0/2738661598 conn(0x7f5b940fc550 msgr2=0x7f5b940fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:19.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.977+0000 7f5b98b31640 1 -- 172.21.15.53:0/2738661598 shutdown_connections 2024-09-19T06:16:19.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.977+0000 7f5b98b31640 1 -- 172.21.15.53:0/2738661598 wait complete. 2024-09-19T06:16:19.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.977+0000 7f5b98b31640 1 Processor -- start 2024-09-19T06:16:19.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.978+0000 7f5b98b31640 1 -- start start 2024-09-19T06:16:19.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.978+0000 7f5b98b31640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b941007f0 0x7f5b94196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:19.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.978+0000 7f5b98b31640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b941067f0 0x7f5b94196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:19.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.978+0000 7f5b98b31640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9410dc90 con 0x7f5b941067f0 2024-09-19T06:16:19.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.978+0000 7f5b98b31640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9410db10 con 0x7f5b941007f0 2024-09-19T06:16:19.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.978+0000 7f5b91d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b941067f0 0x7f5b94196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:19.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.978+0000 7f5b91d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b941067f0 0x7f5b94196b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34358/0 (socket says 172.21.15.53:34358) 2024-09-19T06:16:19.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.978+0000 7f5b91d9b640 1 -- 172.21.15.53:0/3449551596 learned_addr learned my addr 172.21.15.53:0/3449551596 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:19.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.978+0000 7f5b9259c640 1 --2- 172.21.15.53:0/3449551596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b941007f0 0x7f5b94196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:19.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.978+0000 7f5b91d9b640 1 -- 172.21.15.53:0/3449551596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b941007f0 msgr2=0x7f5b94196640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:19.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.978+0000 7f5b91d9b640 1 --2- 172.21.15.53:0/3449551596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b941007f0 0x7f5b94196640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:19.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.978+0000 7f5b91d9b640 1 -- 172.21.15.53:0/3449551596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b941970c0 con 0x7f5b941067f0 2024-09-19T06:16:19.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.978+0000 7f5b91d9b640 1 --2- 172.21.15.53:0/3449551596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b941067f0 0x7f5b94196b80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800b420 tx=0x7f5b8800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:19.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.979+0000 7f5b837fe640 1 -- 172.21.15.53:0/3449551596 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5b88011630 con 0x7f5b941067f0 2024-09-19T06:16:19.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.979+0000 7f5b837fe640 1 -- 172.21.15.53:0/3449551596 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b88016070 con 0x7f5b941067f0 2024-09-19T06:16:19.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.979+0000 7f5b98b31640 1 -- 172.21.15.53:0/3449551596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b941973b0 con 0x7f5b941067f0 2024-09-19T06:16:19.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.979+0000 7f5b837fe640 1 -- 172.21.15.53:0/3449551596 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5b88010c90 con 0x7f5b941067f0 2024-09-19T06:16:19.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.979+0000 7f5b98b31640 1 -- 172.21.15.53:0/3449551596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b941a3e80 con 0x7f5b941067f0 2024-09-19T06:16:19.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.980+0000 7f5b98b31640 1 -- 172.21.15.53:0/3449551596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b94101f30 con 0x7f5b941067f0 2024-09-19T06:16:19.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.980+0000 7f5b837fe640 1 -- 172.21.15.53:0/3449551596 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f5b880117d0 con 0x7f5b941067f0 2024-09-19T06:16:19.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.981+0000 7f5b837fe640 1 --2- 172.21.15.53:0/3449551596 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f5b7c06e300 0x7f5b7c0707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:19.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.981+0000 7f5b9259c640 1 --2- 172.21.15.53:0/3449551596 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f5b7c06e300 0x7f5b7c0707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:19.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.981+0000 7f5b837fe640 1 -- 172.21.15.53:0/3449551596 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5502+0+0 (secure 0 0 0) 0x7f5b8808d950 con 0x7f5b941067f0 2024-09-19T06:16:19.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.981+0000 7f5b9259c640 1 --2- 172.21.15.53:0/3449551596 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f5b7c06e300 0x7f5b7c0707c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5b84000c00 tx=0x7f5b84005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:19.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:19.985+0000 7f5b837fe640 1 -- 172.21.15.53:0/3449551596 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5b8805b280 con 0x7f5b941067f0 2024-09-19T06:16:20.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:20.159+0000 7f5b98b31640 1 -- 172.21.15.53:0/3449551596 --> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5b94062630 con 0x7f5b7c06e300 2024-09-19T06:16:20.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:20.160+0000 7f5b837fe640 1 -- 172.21.15.53:0/3449551596 <== mgr.14217 v2:172.21.15.53:6800/3884355752 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+14526 (secure 0 0 0) 0x7f5b94062630 con 0x7f5b7c06e300 2024-09-19T06:16:20.162 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:16:20.162 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-19T06:16:20.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:20.162+0000 7f5b98b31640 1 -- 172.21.15.53:0/3449551596 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f5b7c06e300 msgr2=0x7f5b7c0707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:20.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:20.162+0000 7f5b98b31640 1 --2- 172.21.15.53:0/3449551596 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f5b7c06e300 0x7f5b7c0707c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5b84000c00 tx=0x7f5b84005a20 comp rx=0 tx=0).stop 2024-09-19T06:16:20.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:20.162+0000 7f5b98b31640 1 -- 172.21.15.53:0/3449551596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b941067f0 msgr2=0x7f5b94196b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:20.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:20.162+0000 7f5b98b31640 1 --2- 172.21.15.53:0/3449551596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b941067f0 0x7f5b94196b80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800b420 tx=0x7f5b8800b8e0 comp rx=0 tx=0).stop 2024-09-19T06:16:20.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:20.162+0000 7f5b98b31640 1 -- 172.21.15.53:0/3449551596 shutdown_connections 2024-09-19T06:16:20.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:20.162+0000 7f5b98b31640 1 --2- 172.21.15.53:0/3449551596 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f5b7c06e300 0x7f5b7c0707c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:20.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:20.162+0000 7f5b98b31640 1 --2- 172.21.15.53:0/3449551596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b941067f0 0x7f5b94196b80 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:20.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:20.162+0000 7f5b98b31640 1 --2- 172.21.15.53:0/3449551596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b941007f0 0x7f5b94196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:20.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:20.162+0000 7f5b98b31640 1 -- 172.21.15.53:0/3449551596 >> 172.21.15.53:0/3449551596 conn(0x7f5b940fc550 msgr2=0x7f5b940fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:20.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:20.162+0000 7f5b98b31640 1 -- 172.21.15.53:0/3449551596 shutdown_connections 2024-09-19T06:16:20.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:20.162+0000 7f5b98b31640 1 -- 172.21.15.53:0/3449551596 wait complete. 2024-09-19T06:16:20.845 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2024-09-19T06:16:18.775584+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":749961216,"kb_used":239512,"kb_used_data":6604,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":749721704,"statfs":{"total":767960285184,"available":767715024896,"internally_reserved":0,"allocated":6762496,"data_stored":3783584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002357"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":214,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-19T06:15:51.825126+0000","last_change":"2024-09-19T06:15:51.824758+0000","last_active":"2024-09-19T06:15:51.825126+0000","last_peered":"2024-09-19T06:15:51.825126+0000","last_clean":"2024-09-19T06:15:51.825126+0000","last_became_active":"2024-09-19T06:15:51.824328+0000","last_became_peered":"2024-09-19T06:15:51.824328+0000","last_unstale":"2024-09-19T06:15:51.825126+0000","last_undegraded":"2024-09-19T06:15:51.825126+0000","last_fullsized":"2024-09-19T06:15:51.825126+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:13:39.618158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:13:39.618158+0000","last_clean_scrub_stamp":"2024-09-19T06:13:39.618158+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:53:49.590551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":56,"seq":240518168586,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28220,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716932,"statfs":{"total":95995035648,"available":95966138368,"internally_reserved":0,"allocated":1216512,"data_stored":841928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462926,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28224,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716928,"statfs":{"total":95995035648,"available":95966134272,"internally_reserved":0,"allocated":1216512,"data_stored":841928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724563,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27640,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717512,"statfs":{"total":95995035648,"available":95966732288,"internally_reserved":0,"allocated":622592,"data_stored":251560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986201,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27644,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717508,"statfs":{"total":95995035648,"available":95966728192,"internally_reserved":0,"allocated":622592,"data_stored":251560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149725,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":31796,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713356,"statfs":{"total":95995035648,"available":95962476544,"internally_reserved":0,"allocated":622592,"data_stored":251560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378658,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":31804,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713348,"statfs":{"total":95995035648,"available":95962468352,"internally_reserved":0,"allocated":622592,"data_stored":251560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542185,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":31800,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713352,"statfs":{"total":95995035648,"available":95962472448,"internally_reserved":0,"allocated":622592,"data_stored":251560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705710,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32384,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712768,"statfs":{"total":95995035648,"available":95961874432,"internally_reserved":0,"allocated":1216512,"data_stored":841928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T06:16:20.845 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T06:16:20.846 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-19T06:16:20.846 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-19T06:16:20.846 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-19T06:16:21.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:20 smithi053 ceph-mon[27968]: from='client.14474 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:16:21.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:20 smithi053 ceph-mon[27968]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:21.109 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:21.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:20 smithi155 ceph-mon[35292]: from='client.14474 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:16:21.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:20 smithi155 ceph-mon[35292]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:21.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.516+0000 7f9803f4d640 1 -- 172.21.15.53:0/2328210760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97fc107900 msgr2=0x7f97fc107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:21.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.516+0000 7f9803f4d640 1 --2- 172.21.15.53:0/2328210760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97fc107900 0x7f97fc107ce0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f97ec009c20 tx=0x7f97ec02a970 comp rx=0 tx=0).stop 2024-09-19T06:16:21.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.516+0000 7f9803f4d640 1 -- 172.21.15.53:0/2328210760 shutdown_connections 2024-09-19T06:16:21.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.516+0000 7f9803f4d640 1 --2- 172.21.15.53:0/2328210760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97fc101900 0x7f97fc101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:21.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.516+0000 7f9803f4d640 1 --2- 172.21.15.53:0/2328210760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97fc107900 0x7f97fc107ce0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:21.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.516+0000 7f9803f4d640 1 -- 172.21.15.53:0/2328210760 >> 172.21.15.53:0/2328210760 conn(0x7f97fc0fd600 msgr2=0x7f97fc0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:21.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.516+0000 7f9803f4d640 1 -- 172.21.15.53:0/2328210760 shutdown_connections 2024-09-19T06:16:21.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.516+0000 7f9803f4d640 1 -- 172.21.15.53:0/2328210760 wait complete. 2024-09-19T06:16:21.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.517+0000 7f9803f4d640 1 Processor -- start 2024-09-19T06:16:21.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.517+0000 7f9803f4d640 1 -- start start 2024-09-19T06:16:21.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.517+0000 7f9803f4d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97fc101900 0x7f97fc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:21.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.517+0000 7f9803f4d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97fc107900 0x7f97fc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:21.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.517+0000 7f9803f4d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc10ed40 con 0x7f97fc101900 2024-09-19T06:16:21.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.517+0000 7f9803f4d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc10ebc0 con 0x7f97fc107900 2024-09-19T06:16:21.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.518+0000 7f9801ce9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97fc101900 0x7f97fc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:21.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.518+0000 7f9801ce9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97fc101900 0x7f97fc0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51748/0 (socket says 172.21.15.53:51748) 2024-09-19T06:16:21.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.518+0000 7f9801ce9640 1 -- 172.21.15.53:0/1842642601 learned_addr learned my addr 172.21.15.53:0/1842642601 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:21.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.518+0000 7f9801ce9640 1 -- 172.21.15.53:0/1842642601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97fc107900 msgr2=0x7f97fc074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:21.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.518+0000 7f9801ce9640 1 --2- 172.21.15.53:0/1842642601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97fc107900 0x7f97fc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:21.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.518+0000 7f9801ce9640 1 -- 172.21.15.53:0/1842642601 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97fc074f70 con 0x7f97fc101900 2024-09-19T06:16:21.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.518+0000 7f9801ce9640 1 --2- 172.21.15.53:0/1842642601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97fc101900 0x7f97fc0744f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f97ec005cc0 tx=0x7f97ec02f660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:21.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.518+0000 7f97faffd640 1 -- 172.21.15.53:0/1842642601 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f97ec047070 con 0x7f97fc101900 2024-09-19T06:16:21.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.518+0000 7f9803f4d640 1 -- 172.21.15.53:0/1842642601 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97fc075200 con 0x7f97fc101900 2024-09-19T06:16:21.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.518+0000 7f97faffd640 1 -- 172.21.15.53:0/1842642601 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97ec003e20 con 0x7f97fc101900 2024-09-19T06:16:21.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.518+0000 7f97faffd640 1 -- 172.21.15.53:0/1842642601 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f97ec034070 con 0x7f97fc101900 2024-09-19T06:16:21.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.518+0000 7f9803f4d640 1 -- 172.21.15.53:0/1842642601 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97fc1a3cb0 con 0x7f97fc101900 2024-09-19T06:16:21.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.519+0000 7f9803f4d640 1 -- 172.21.15.53:0/1842642601 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97fc103040 con 0x7f97fc101900 2024-09-19T06:16:21.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.520+0000 7f97faffd640 1 -- 172.21.15.53:0/1842642601 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f97ec04a070 con 0x7f97fc101900 2024-09-19T06:16:21.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.520+0000 7f97faffd640 1 --2- 172.21.15.53:0/1842642601 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f97e406e2b0 0x7f97e4070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:21.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.520+0000 7f97faffd640 1 -- 172.21.15.53:0/1842642601 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5502+0+0 (secure 0 0 0) 0x7f97ec0af020 con 0x7f97fc101900 2024-09-19T06:16:21.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.520+0000 7f98014e8640 1 --2- 172.21.15.53:0/1842642601 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f97e406e2b0 0x7f97e4070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:21.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.521+0000 7f98014e8640 1 --2- 172.21.15.53:0/1842642601 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f97e406e2b0 0x7f97e4070770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f97f0005fd0 tx=0x7f97f0005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:21.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.525+0000 7f97faffd640 1 -- 172.21.15.53:0/1842642601 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f97ec03f070 con 0x7f97fc101900 2024-09-19T06:16:21.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.742+0000 7f9803f4d640 1 -- 172.21.15.53:0/1842642601 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f97fc109e70 con 0x7f97fc101900 2024-09-19T06:16:21.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.742+0000 7f97faffd640 1 -- 172.21.15.53:0/1842642601 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f97ec07c8d0 con 0x7f97fc101900 2024-09-19T06:16:21.744 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:16:21.744 INFO:teuthology.orchestra.run.smithi053.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T06:16:21.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.744+0000 7f9803f4d640 1 -- 172.21.15.53:0/1842642601 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f97e406e2b0 msgr2=0x7f97e4070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:21.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.744+0000 7f9803f4d640 1 --2- 172.21.15.53:0/1842642601 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f97e406e2b0 0x7f97e4070770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f97f0005fd0 tx=0x7f97f0005850 comp rx=0 tx=0).stop 2024-09-19T06:16:21.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.744+0000 7f9803f4d640 1 -- 172.21.15.53:0/1842642601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97fc101900 msgr2=0x7f97fc0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:21.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.744+0000 7f9803f4d640 1 --2- 172.21.15.53:0/1842642601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97fc101900 0x7f97fc0744f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f97ec005cc0 tx=0x7f97ec02f660 comp rx=0 tx=0).stop 2024-09-19T06:16:21.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.744+0000 7f9803f4d640 1 -- 172.21.15.53:0/1842642601 shutdown_connections 2024-09-19T06:16:21.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.744+0000 7f9803f4d640 1 --2- 172.21.15.53:0/1842642601 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f97e406e2b0 0x7f97e4070770 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:21.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.744+0000 7f9803f4d640 1 --2- 172.21.15.53:0/1842642601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97fc107900 0x7f97fc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:21.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.744+0000 7f9803f4d640 1 --2- 172.21.15.53:0/1842642601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97fc101900 0x7f97fc0744f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:21.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.744+0000 7f9803f4d640 1 -- 172.21.15.53:0/1842642601 >> 172.21.15.53:0/1842642601 conn(0x7f97fc0fd600 msgr2=0x7f97fc0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:21.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.745+0000 7f9803f4d640 1 -- 172.21.15.53:0/1842642601 shutdown_connections 2024-09-19T06:16:21.748 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:21.745+0000 7f9803f4d640 1 -- 172.21.15.53:0/1842642601 wait complete. 2024-09-19T06:16:22.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:21 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/1842642601' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T06:16:22.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:21 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/1842642601' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T06:16:22.298 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-19T06:16:22.298 INFO:tasks.cephadm:Setup complete, yielding 2024-09-19T06:16:22.298 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T06:16:22.309 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi053.front.sepia.ceph.com 2024-09-19T06:16:22.309 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-19T06:16:22.570 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:22.965 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:22 smithi053 ceph-mon[27968]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:23.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.046+0000 7efe903eb640 1 -- 172.21.15.53:0/4157789945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe88104d20 msgr2=0x7efe88107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:23.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.046+0000 7efe903eb640 1 --2- 172.21.15.53:0/4157789945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe88104d20 0x7efe88107110 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efe780098b0 tx=0x7efe7802a4d0 comp rx=0 tx=0).stop 2024-09-19T06:16:23.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.046+0000 7efe903eb640 1 -- 172.21.15.53:0/4157789945 shutdown_connections 2024-09-19T06:16:23.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.046+0000 7efe903eb640 1 --2- 172.21.15.53:0/4157789945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe88104d20 0x7efe88107110 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:23.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.046+0000 7efe903eb640 1 --2- 172.21.15.53:0/4157789945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe881023f0 0x7efe881047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:23.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.046+0000 7efe903eb640 1 -- 172.21.15.53:0/4157789945 >> 172.21.15.53:0/4157789945 conn(0x7efe880fc010 msgr2=0x7efe880fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:23.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.046+0000 7efe903eb640 1 -- 172.21.15.53:0/4157789945 shutdown_connections 2024-09-19T06:16:23.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.046+0000 7efe903eb640 1 -- 172.21.15.53:0/4157789945 wait complete. 2024-09-19T06:16:23.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.047+0000 7efe903eb640 1 Processor -- start 2024-09-19T06:16:23.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.047+0000 7efe903eb640 1 -- start start 2024-09-19T06:16:23.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.047+0000 7efe903eb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe881023f0 0x7efe881974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:23.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.047+0000 7efe903eb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe88104d20 0x7efe881979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:23.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.047+0000 7efe903eb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efe8810a700 con 0x7efe881023f0 2024-09-19T06:16:23.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.047+0000 7efe903eb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efe8810a580 con 0x7efe88104d20 2024-09-19T06:16:23.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.047+0000 7efe8e187640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe881023f0 0x7efe881974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:23.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.047+0000 7efe8e187640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe881023f0 0x7efe881974a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51768/0 (socket says 172.21.15.53:51768) 2024-09-19T06:16:23.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.047+0000 7efe8e187640 1 -- 172.21.15.53:0/1453663151 learned_addr learned my addr 172.21.15.53:0/1453663151 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:23.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.047+0000 7efe8d986640 1 --2- 172.21.15.53:0/1453663151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe88104d20 0x7efe881979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:23.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.047+0000 7efe8e187640 1 -- 172.21.15.53:0/1453663151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe88104d20 msgr2=0x7efe881979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:23.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.047+0000 7efe8e187640 1 --2- 172.21.15.53:0/1453663151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe88104d20 0x7efe881979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:23.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.047+0000 7efe8e187640 1 -- 172.21.15.53:0/1453663151 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe88197fd0 con 0x7efe881023f0 2024-09-19T06:16:23.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.047+0000 7efe8d986640 1 --2- 172.21.15.53:0/1453663151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe88104d20 0x7efe881979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:16:23.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.048+0000 7efe8e187640 1 --2- 172.21.15.53:0/1453663151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe881023f0 0x7efe881974a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efe8400c810 tx=0x7efe8400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:23.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.048+0000 7efe7f7fe640 1 -- 172.21.15.53:0/1453663151 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efe84004020 con 0x7efe881023f0 2024-09-19T06:16:23.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.048+0000 7efe7f7fe640 1 -- 172.21.15.53:0/1453663151 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe840041c0 con 0x7efe881023f0 2024-09-19T06:16:23.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.048+0000 7efe7f7fe640 1 -- 172.21.15.53:0/1453663151 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efe840047a0 con 0x7efe881023f0 2024-09-19T06:16:23.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.048+0000 7efe903eb640 1 -- 172.21.15.53:0/1453663151 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe8819ca40 con 0x7efe881023f0 2024-09-19T06:16:23.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.048+0000 7efe903eb640 1 -- 172.21.15.53:0/1453663151 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe8819d020 con 0x7efe881023f0 2024-09-19T06:16:23.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.049+0000 7efe903eb640 1 -- 172.21.15.53:0/1453663151 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe8810ae50 con 0x7efe881023f0 2024-09-19T06:16:23.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.049+0000 7efe7f7fe640 1 -- 172.21.15.53:0/1453663151 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7efe840079b0 con 0x7efe881023f0 2024-09-19T06:16:23.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.050+0000 7efe7f7fe640 1 --2- 172.21.15.53:0/1453663151 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7efe7406e310 0x7efe740707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:23.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.050+0000 7efe7f7fe640 1 -- 172.21.15.53:0/1453663151 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5502+0+0 (secure 0 0 0) 0x7efe8408e010 con 0x7efe881023f0 2024-09-19T06:16:23.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.050+0000 7efe8d986640 1 --2- 172.21.15.53:0/1453663151 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7efe7406e310 0x7efe740707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:23.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.050+0000 7efe8d986640 1 --2- 172.21.15.53:0/1453663151 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7efe7406e310 0x7efe740707d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efe78006010 tx=0x7efe78004e20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:23.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.054+0000 7efe7f7fe640 1 -- 172.21.15.53:0/1453663151 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efe84014030 con 0x7efe881023f0 2024-09-19T06:16:23.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:22 smithi155 ceph-mon[35292]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:23.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.249+0000 7efe903eb640 1 -- 172.21.15.53:0/1453663151 --> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7efe881055a0 con 0x7efe7406e310 2024-09-19T06:16:23.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.250+0000 7efe7f7fe640 1 -- 172.21.15.53:0/1453663151 <== mgr.14217 v2:172.21.15.53:6800/3884355752 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7efe881055a0 con 0x7efe7406e310 2024-09-19T06:16:23.251 INFO:teuthology.orchestra.run.smithi053.stdout:Backend: cephadm 2024-09-19T06:16:23.252 INFO:teuthology.orchestra.run.smithi053.stdout:Available: Yes 2024-09-19T06:16:23.252 INFO:teuthology.orchestra.run.smithi053.stdout:Paused: No 2024-09-19T06:16:23.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.253+0000 7efe903eb640 1 -- 172.21.15.53:0/1453663151 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7efe7406e310 msgr2=0x7efe740707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:23.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.253+0000 7efe903eb640 1 --2- 172.21.15.53:0/1453663151 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7efe7406e310 0x7efe740707d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efe78006010 tx=0x7efe78004e20 comp rx=0 tx=0).stop 2024-09-19T06:16:23.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.253+0000 7efe903eb640 1 -- 172.21.15.53:0/1453663151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe881023f0 msgr2=0x7efe881974a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:23.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.253+0000 7efe903eb640 1 --2- 172.21.15.53:0/1453663151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe881023f0 0x7efe881974a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efe8400c810 tx=0x7efe8400ccd0 comp rx=0 tx=0).stop 2024-09-19T06:16:23.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.253+0000 7efe903eb640 1 -- 172.21.15.53:0/1453663151 shutdown_connections 2024-09-19T06:16:23.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.253+0000 7efe903eb640 1 --2- 172.21.15.53:0/1453663151 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7efe7406e310 0x7efe740707d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:23.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.253+0000 7efe903eb640 1 --2- 172.21.15.53:0/1453663151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe88104d20 0x7efe881979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:23.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.253+0000 7efe903eb640 1 --2- 172.21.15.53:0/1453663151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe881023f0 0x7efe881974a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:23.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.253+0000 7efe903eb640 1 -- 172.21.15.53:0/1453663151 >> 172.21.15.53:0/1453663151 conn(0x7efe880fc010 msgr2=0x7efe880fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:23.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.253+0000 7efe903eb640 1 -- 172.21.15.53:0/1453663151 shutdown_connections 2024-09-19T06:16:23.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:23.253+0000 7efe903eb640 1 -- 172.21.15.53:0/1453663151 wait complete. 2024-09-19T06:16:23.831 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-19T06:16:23.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:23 smithi155 ceph-mon[35292]: from='client.14482 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:16:24.091 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:24.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:23 smithi053 ceph-mon[27968]: from='client.14482 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:16:24.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.586+0000 7f5235081640 1 -- 172.21.15.53:0/2296329017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5230102b30 msgr2=0x7f5230104fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:24.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.586+0000 7f5235081640 1 --2- 172.21.15.53:0/2296329017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5230102b30 0x7f5230104fa0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5224009880 tx=0x7f522402a4a0 comp rx=0 tx=0).stop 2024-09-19T06:16:24.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.586+0000 7f5235081640 1 -- 172.21.15.53:0/2296329017 shutdown_connections 2024-09-19T06:16:24.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.586+0000 7f5235081640 1 --2- 172.21.15.53:0/2296329017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5230102b30 0x7f5230104fa0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:24.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.586+0000 7f5235081640 1 --2- 172.21.15.53:0/2296329017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52301001e0 0x7f52301025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:24.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.586+0000 7f5235081640 1 -- 172.21.15.53:0/2296329017 >> 172.21.15.53:0/2296329017 conn(0x7f52300fc010 msgr2=0x7f52300fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:24.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.586+0000 7f5235081640 1 -- 172.21.15.53:0/2296329017 shutdown_connections 2024-09-19T06:16:24.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.586+0000 7f5235081640 1 -- 172.21.15.53:0/2296329017 wait complete. 2024-09-19T06:16:24.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.587+0000 7f5235081640 1 Processor -- start 2024-09-19T06:16:24.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.587+0000 7f5235081640 1 -- start start 2024-09-19T06:16:24.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.587+0000 7f5235081640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52301001e0 0x7f523019daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:24.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.587+0000 7f5235081640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5230102b30 0x7f523019e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:24.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.587+0000 7f5235081640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5230110d70 con 0x7f52301001e0 2024-09-19T06:16:24.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.587+0000 7f5235081640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5230110bf0 con 0x7f5230102b30 2024-09-19T06:16:24.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.588+0000 7f522ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52301001e0 0x7f523019daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:24.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.588+0000 7f522ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52301001e0 0x7f523019daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51788/0 (socket says 172.21.15.53:51788) 2024-09-19T06:16:24.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.588+0000 7f522ed9d640 1 -- 172.21.15.53:0/3337703545 learned_addr learned my addr 172.21.15.53:0/3337703545 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:24.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.588+0000 7f522e59c640 1 --2- 172.21.15.53:0/3337703545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5230102b30 0x7f523019e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:24.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.588+0000 7f522ed9d640 1 -- 172.21.15.53:0/3337703545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5230102b30 msgr2=0x7f523019e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:24.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.588+0000 7f522ed9d640 1 --2- 172.21.15.53:0/3337703545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5230102b30 0x7f523019e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:24.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.588+0000 7f522ed9d640 1 -- 172.21.15.53:0/3337703545 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523019e620 con 0x7f52301001e0 2024-09-19T06:16:24.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.588+0000 7f522ed9d640 1 --2- 172.21.15.53:0/3337703545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52301001e0 0x7f523019daf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f522000b420 tx=0x7f522000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:24.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.588+0000 7f5217fff640 1 -- 172.21.15.53:0/3337703545 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5220011630 con 0x7f52301001e0 2024-09-19T06:16:24.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.588+0000 7f5235081640 1 -- 172.21.15.53:0/3337703545 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52301a3090 con 0x7f52301001e0 2024-09-19T06:16:24.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.588+0000 7f5217fff640 1 -- 172.21.15.53:0/3337703545 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5220016070 con 0x7f52301001e0 2024-09-19T06:16:24.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.588+0000 7f5217fff640 1 -- 172.21.15.53:0/3337703545 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5220010c90 con 0x7f52301001e0 2024-09-19T06:16:24.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.588+0000 7f5235081640 1 -- 172.21.15.53:0/3337703545 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52301a3670 con 0x7f52301001e0 2024-09-19T06:16:24.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.589+0000 7f5235081640 1 -- 172.21.15.53:0/3337703545 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52301114c0 con 0x7f52301001e0 2024-09-19T06:16:24.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.590+0000 7f5217fff640 1 -- 172.21.15.53:0/3337703545 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f52200117d0 con 0x7f52301001e0 2024-09-19T06:16:24.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.590+0000 7f5217fff640 1 --2- 172.21.15.53:0/3337703545 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f521806e2b0 0x7f5218070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:24.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.591+0000 7f5217fff640 1 -- 172.21.15.53:0/3337703545 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5502+0+0 (secure 0 0 0) 0x7f522008dab0 con 0x7f52301001e0 2024-09-19T06:16:24.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.591+0000 7f522e59c640 1 --2- 172.21.15.53:0/3337703545 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f521806e2b0 0x7f5218070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:24.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.591+0000 7f522e59c640 1 --2- 172.21.15.53:0/3337703545 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f521806e2b0 0x7f5218070770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f523019efa0 tx=0x7f5224004e20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:24.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.595+0000 7f5217fff640 1 -- 172.21.15.53:0/3337703545 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f522005b2c0 con 0x7f52301001e0 2024-09-19T06:16:24.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.781+0000 7f5235081640 1 -- 172.21.15.53:0/3337703545 --> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f52301033b0 con 0x7f521806e2b0 2024-09-19T06:16:24.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.790+0000 7f5217fff640 1 -- 172.21.15.53:0/3337703545 <== mgr.14217 v2:172.21.15.53:6800/3884355752 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3388 (secure 0 0 0) 0x7f52301033b0 con 0x7f521806e2b0 2024-09-19T06:16:24.792 INFO:teuthology.orchestra.run.smithi053.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-19T06:16:24.792 INFO:teuthology.orchestra.run.smithi053.stdout:alertmanager.smithi053 smithi053 *:9093,9094 running (4m) 2m ago 5m 26.2M - 0.25.0 c8568f914cd2 18430d118311 2024-09-19T06:16:24.792 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-exporter.smithi053 smithi053 running (5m) 2m ago 5m 7922k - 19.1.1-361-g2f599a46 d2baa855c067 f2d7b3156da7 2024-09-19T06:16:24.792 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-exporter.smithi155 smithi155 running (4m) 40s ago 4m 5293k - 19.1.1-361-g2f599a46 d2baa855c067 85703f975126 2024-09-19T06:16:24.792 INFO:teuthology.orchestra.run.smithi053.stdout:crash.smithi053 smithi053 running (5m) 2m ago 5m 6895k - 19.1.1-361-g2f599a46 d2baa855c067 b3820106668f 2024-09-19T06:16:24.792 INFO:teuthology.orchestra.run.smithi053.stdout:crash.smithi155 smithi155 running (4m) 40s ago 4m 6895k - 19.1.1-361-g2f599a46 d2baa855c067 4971586b3139 2024-09-19T06:16:24.792 INFO:teuthology.orchestra.run.smithi053.stdout:grafana.smithi053 smithi053 *:3000 running (3m) 2m ago 5m 73.9M - 9.4.12 2bacad6d85d8 579369b69789 2024-09-19T06:16:24.793 INFO:teuthology.orchestra.run.smithi053.stdout:mgr.smithi053.trakvg smithi053 *:9283,8765,8443 running (6m) 2m ago 6m 494M - 19.1.1-361-g2f599a46 d2baa855c067 59e4ebb59d3a 2024-09-19T06:16:24.793 INFO:teuthology.orchestra.run.smithi053.stdout:mgr.smithi155.piymgv smithi155 *:8443,9283,8765 running (4m) 40s ago 4m 440M - 19.1.1-361-g2f599a46 d2baa855c067 e587efcb1cff 2024-09-19T06:16:24.793 INFO:teuthology.orchestra.run.smithi053.stdout:mon.smithi053 smithi053 running (6m) 2m ago 6m 48.8M 2048M 19.1.1-361-g2f599a46 d2baa855c067 3a3c39f8318d 2024-09-19T06:16:24.793 INFO:teuthology.orchestra.run.smithi053.stdout:mon.smithi155 smithi155 running (4m) 40s ago 4m 45.0M 2048M 19.1.1-361-g2f599a46 d2baa855c067 043f44f8feb0 2024-09-19T06:16:24.793 INFO:teuthology.orchestra.run.smithi053.stdout:node-exporter.smithi053 smithi053 *:9100 running (5m) 2m ago 5m 15.7M - 1.5.0 0da6a335fe13 56854f21717c 2024-09-19T06:16:24.793 INFO:teuthology.orchestra.run.smithi053.stdout:node-exporter.smithi155 smithi155 *:9100 running (4m) 40s ago 4m 15.1M - 1.5.0 0da6a335fe13 7aab4bfea7ca 2024-09-19T06:16:24.793 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0 smithi053 running (3m) 2m ago 3m 47.7M 2961M 19.1.1-361-g2f599a46 d2baa855c067 e2d04a79b637 2024-09-19T06:16:24.793 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1 smithi053 running (3m) 2m ago 3m 45.4M 2961M 19.1.1-361-g2f599a46 d2baa855c067 771e86d6a506 2024-09-19T06:16:24.793 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2 smithi053 running (2m) 2m ago 2m 44.9M 2961M 19.1.1-361-g2f599a46 d2baa855c067 18e322e6f311 2024-09-19T06:16:24.793 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3 smithi053 running (2m) 2m ago 2m 12.1M 2961M 19.1.1-361-g2f599a46 d2baa855c067 ba2d56fafe1e 2024-09-19T06:16:24.793 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4 smithi155 running (2m) 40s ago 2m 44.5M 3728M 19.1.1-361-g2f599a46 d2baa855c067 7e08cad4b619 2024-09-19T06:16:24.793 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5 smithi155 running (95s) 40s ago 95s 43.5M 3728M 19.1.1-361-g2f599a46 d2baa855c067 1c97b1724d7e 2024-09-19T06:16:24.793 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6 smithi155 running (70s) 40s ago 70s 36.2M 3728M 19.1.1-361-g2f599a46 d2baa855c067 8a895f8b6ce6 2024-09-19T06:16:24.793 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7 smithi155 running (45s) 40s ago 45s 11.0M 3728M 19.1.1-361-g2f599a46 d2baa855c067 38847310f499 2024-09-19T06:16:24.794 INFO:teuthology.orchestra.run.smithi053.stdout:prometheus.smithi053 smithi053 *:9095 running (3m) 2m ago 5m 44.9M - 2.43.0 a07b618ecd1d a7a1edd1fa9c 2024-09-19T06:16:24.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.792+0000 7f5235081640 1 -- 172.21.15.53:0/3337703545 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f521806e2b0 msgr2=0x7f5218070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:24.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.793+0000 7f5235081640 1 --2- 172.21.15.53:0/3337703545 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f521806e2b0 0x7f5218070770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f523019efa0 tx=0x7f5224004e20 comp rx=0 tx=0).stop 2024-09-19T06:16:24.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.793+0000 7f5235081640 1 -- 172.21.15.53:0/3337703545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52301001e0 msgr2=0x7f523019daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:24.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.793+0000 7f5235081640 1 --2- 172.21.15.53:0/3337703545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52301001e0 0x7f523019daf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f522000b420 tx=0x7f522000b8e0 comp rx=0 tx=0).stop 2024-09-19T06:16:24.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.793+0000 7f5235081640 1 -- 172.21.15.53:0/3337703545 shutdown_connections 2024-09-19T06:16:24.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.793+0000 7f5235081640 1 --2- 172.21.15.53:0/3337703545 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f521806e2b0 0x7f5218070770 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:24.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.793+0000 7f5235081640 1 --2- 172.21.15.53:0/3337703545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5230102b30 0x7f523019e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:24.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.793+0000 7f5235081640 1 --2- 172.21.15.53:0/3337703545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52301001e0 0x7f523019daf0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:24.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.793+0000 7f5235081640 1 -- 172.21.15.53:0/3337703545 >> 172.21.15.53:0/3337703545 conn(0x7f52300fc010 msgr2=0x7f52300fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:24.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.793+0000 7f5235081640 1 -- 172.21.15.53:0/3337703545 shutdown_connections 2024-09-19T06:16:24.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:24.793+0000 7f5235081640 1 -- 172.21.15.53:0/3337703545 wait complete. 2024-09-19T06:16:25.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:24 smithi053 ceph-mon[27968]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:25.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:24 smithi155 ceph-mon[35292]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:25.390 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-19T06:16:25.648 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:26.047 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:25 smithi053 ceph-mon[27968]: from='client.14486 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:16:26.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.129+0000 7fe003942640 1 -- 172.21.15.53:0/2876948972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdffc104d80 msgr2=0x7fdffc107170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:26.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.129+0000 7fe003942640 1 --2- 172.21.15.53:0/2876948972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdffc104d80 0x7fdffc107170 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009880 tx=0x7fdfec02a4a0 comp rx=0 tx=0).stop 2024-09-19T06:16:26.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.129+0000 7fe003942640 1 -- 172.21.15.53:0/2876948972 shutdown_connections 2024-09-19T06:16:26.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.129+0000 7fe003942640 1 --2- 172.21.15.53:0/2876948972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdffc104d80 0x7fdffc107170 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:26.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.129+0000 7fe003942640 1 --2- 172.21.15.53:0/2876948972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdffc102450 0x7fdffc104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:26.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.129+0000 7fe003942640 1 -- 172.21.15.53:0/2876948972 >> 172.21.15.53:0/2876948972 conn(0x7fdffc0fc010 msgr2=0x7fdffc0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:26.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.129+0000 7fe003942640 1 -- 172.21.15.53:0/2876948972 shutdown_connections 2024-09-19T06:16:26.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.129+0000 7fe003942640 1 -- 172.21.15.53:0/2876948972 wait complete. 2024-09-19T06:16:26.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.130+0000 7fe003942640 1 Processor -- start 2024-09-19T06:16:26.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.130+0000 7fe003942640 1 -- start start 2024-09-19T06:16:26.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.130+0000 7fe003942640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdffc102450 0x7fdffc197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:26.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.130+0000 7fe003942640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdffc104d80 0x7fdffc197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:26.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.130+0000 7fe003942640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc10a710 con 0x7fdffc102450 2024-09-19T06:16:26.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.130+0000 7fe003942640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc10a590 con 0x7fdffc104d80 2024-09-19T06:16:26.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.130+0000 7fe0016de640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdffc102450 0x7fdffc197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:26.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.130+0000 7fe0016de640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdffc102450 0x7fdffc197560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51820/0 (socket says 172.21.15.53:51820) 2024-09-19T06:16:26.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.130+0000 7fe0016de640 1 -- 172.21.15.53:0/1845688679 learned_addr learned my addr 172.21.15.53:0/1845688679 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:26.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.130+0000 7fe000edd640 1 --2- 172.21.15.53:0/1845688679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdffc104d80 0x7fdffc197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:26.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.130+0000 7fe0016de640 1 -- 172.21.15.53:0/1845688679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdffc104d80 msgr2=0x7fdffc197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:26.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.130+0000 7fe0016de640 1 --2- 172.21.15.53:0/1845688679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdffc104d80 0x7fdffc197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:26.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.130+0000 7fe0016de640 1 -- 172.21.15.53:0/1845688679 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdffc198090 con 0x7fdffc102450 2024-09-19T06:16:26.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.131+0000 7fe0016de640 1 --2- 172.21.15.53:0/1845688679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdffc102450 0x7fdffc197560 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdff800b420 tx=0x7fdff800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:26.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.131+0000 7fdff27fc640 1 -- 172.21.15.53:0/1845688679 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdff8011630 con 0x7fdffc102450 2024-09-19T06:16:26.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.131+0000 7fdff27fc640 1 -- 172.21.15.53:0/1845688679 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdff8016070 con 0x7fdffc102450 2024-09-19T06:16:26.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.131+0000 7fe003942640 1 -- 172.21.15.53:0/1845688679 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdffc19cb00 con 0x7fdffc102450 2024-09-19T06:16:26.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.131+0000 7fe003942640 1 -- 172.21.15.53:0/1845688679 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdffc19d060 con 0x7fdffc102450 2024-09-19T06:16:26.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.131+0000 7fdff27fc640 1 -- 172.21.15.53:0/1845688679 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdff8010c90 con 0x7fdffc102450 2024-09-19T06:16:26.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.132+0000 7fe003942640 1 -- 172.21.15.53:0/1845688679 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdffc1028b0 con 0x7fdffc102450 2024-09-19T06:16:26.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.133+0000 7fdff27fc640 1 -- 172.21.15.53:0/1845688679 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7fdff80117d0 con 0x7fdffc102450 2024-09-19T06:16:26.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.133+0000 7fdff27fc640 1 --2- 172.21.15.53:0/1845688679 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fdfe806e2b0 0x7fdfe8070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:26.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.133+0000 7fdff27fc640 1 -- 172.21.15.53:0/1845688679 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5502+0+0 (secure 0 0 0) 0x7fdff808dab0 con 0x7fdffc102450 2024-09-19T06:16:26.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.133+0000 7fe000edd640 1 --2- 172.21.15.53:0/1845688679 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fdfe806e2b0 0x7fdfe8070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:26.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.134+0000 7fe000edd640 1 --2- 172.21.15.53:0/1845688679 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fdfe806e2b0 0x7fdfe8070770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdfec000c00 tx=0x7fdfec004e20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:26.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.138+0000 7fdff27fc640 1 -- 172.21.15.53:0/1845688679 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdff805b360 con 0x7fdffc102450 2024-09-19T06:16:26.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:25 smithi155 ceph-mon[35292]: from='client.14486 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:16:26.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.319+0000 7fe003942640 1 -- 172.21.15.53:0/1845688679 --> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fdffc1056a0 con 0x7fdfe806e2b0 2024-09-19T06:16:26.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.324+0000 7fdff27fc640 1 -- 172.21.15.53:0/1845688679 <== mgr.14217 v2:172.21.15.53:6800/3884355752 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+1300 (secure 0 0 0) 0x7fdffc1056a0 con 0x7fdfe806e2b0 2024-09-19T06:16:26.326 INFO:teuthology.orchestra.run.smithi053.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-19T06:16:26.326 INFO:teuthology.orchestra.run.smithi053.stdout:alertmanager ?:9093,9094 1/1 2m ago 6m count:1 2024-09-19T06:16:26.326 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-exporter 2/2 2m ago 6m * 2024-09-19T06:16:26.326 INFO:teuthology.orchestra.run.smithi053.stdout:crash 2/2 2m ago 6m * 2024-09-19T06:16:26.326 INFO:teuthology.orchestra.run.smithi053.stdout:grafana ?:3000 1/1 2m ago 6m count:1 2024-09-19T06:16:26.326 INFO:teuthology.orchestra.run.smithi053.stdout:mgr 2/2 2m ago 6m count:2 2024-09-19T06:16:26.326 INFO:teuthology.orchestra.run.smithi053.stdout:mon 2/2 2m ago 5m smithi053:172.21.15.53=smithi053;smithi155:172.21.15.155=smithi155;count:2 2024-09-19T06:16:26.327 INFO:teuthology.orchestra.run.smithi053.stdout:node-exporter ?:9100 2/2 2m ago 6m * 2024-09-19T06:16:26.327 INFO:teuthology.orchestra.run.smithi053.stdout:osd 8 2m ago - 2024-09-19T06:16:26.327 INFO:teuthology.orchestra.run.smithi053.stdout:prometheus ?:9095 1/1 2m ago 6m count:1 2024-09-19T06:16:26.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.326+0000 7fe003942640 1 -- 172.21.15.53:0/1845688679 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fdfe806e2b0 msgr2=0x7fdfe8070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:26.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.326+0000 7fe003942640 1 --2- 172.21.15.53:0/1845688679 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fdfe806e2b0 0x7fdfe8070770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdfec000c00 tx=0x7fdfec004e20 comp rx=0 tx=0).stop 2024-09-19T06:16:26.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.326+0000 7fe003942640 1 -- 172.21.15.53:0/1845688679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdffc102450 msgr2=0x7fdffc197560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:26.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.326+0000 7fe003942640 1 --2- 172.21.15.53:0/1845688679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdffc102450 0x7fdffc197560 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdff800b420 tx=0x7fdff800b8e0 comp rx=0 tx=0).stop 2024-09-19T06:16:26.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.326+0000 7fe003942640 1 -- 172.21.15.53:0/1845688679 shutdown_connections 2024-09-19T06:16:26.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.326+0000 7fe003942640 1 --2- 172.21.15.53:0/1845688679 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fdfe806e2b0 0x7fdfe8070770 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:26.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.326+0000 7fe003942640 1 --2- 172.21.15.53:0/1845688679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdffc104d80 0x7fdffc197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:26.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.326+0000 7fe003942640 1 --2- 172.21.15.53:0/1845688679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdffc102450 0x7fdffc197560 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:26.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.326+0000 7fe003942640 1 -- 172.21.15.53:0/1845688679 >> 172.21.15.53:0/1845688679 conn(0x7fdffc0fc010 msgr2=0x7fdffc0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:26.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.326+0000 7fe003942640 1 -- 172.21.15.53:0/1845688679 shutdown_connections 2024-09-19T06:16:26.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:26.326+0000 7fe003942640 1 -- 172.21.15.53:0/1845688679 wait complete. 2024-09-19T06:16:26.894 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:26 smithi053 ceph-mon[27968]: from='client.14490 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:16:26.894 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:26 smithi053 ceph-mon[27968]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:26.923 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-19T06:16:27.183 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:27.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:26 smithi155 ceph-mon[35292]: from='client.14490 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:16:27.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:26 smithi155 ceph-mon[35292]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:27.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.580+0000 7f4a75c9d640 1 -- 172.21.15.53:0/1124580236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a70101a60 msgr2=0x7f4a70101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:27.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.580+0000 7f4a75c9d640 1 --2- 172.21.15.53:0/1124580236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a70101a60 0x7f4a70101ee0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4a64009880 tx=0x7f4a6402a4a0 comp rx=0 tx=0).stop 2024-09-19T06:16:27.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.580+0000 7f4a75c9d640 1 -- 172.21.15.53:0/1124580236 shutdown_connections 2024-09-19T06:16:27.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.580+0000 7f4a75c9d640 1 --2- 172.21.15.53:0/1124580236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a70101a60 0x7f4a70101ee0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:27.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.580+0000 7f4a75c9d640 1 --2- 172.21.15.53:0/1124580236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a70100860 0x7f4a70100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:27.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.580+0000 7f4a75c9d640 1 -- 172.21.15.53:0/1124580236 >> 172.21.15.53:0/1124580236 conn(0x7f4a700fc010 msgr2=0x7f4a700fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:27.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.580+0000 7f4a75c9d640 1 -- 172.21.15.53:0/1124580236 shutdown_connections 2024-09-19T06:16:27.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.580+0000 7f4a75c9d640 1 -- 172.21.15.53:0/1124580236 wait complete. 2024-09-19T06:16:27.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.581+0000 7f4a75c9d640 1 Processor -- start 2024-09-19T06:16:27.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.581+0000 7f4a75c9d640 1 -- start start 2024-09-19T06:16:27.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.581+0000 7f4a75c9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a70100860 0x7f4a7006a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:27.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.581+0000 7f4a75c9d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a70101a60 0x7f4a7019c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:27.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.581+0000 7f4a75c9d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4a701095f0 con 0x7f4a70100860 2024-09-19T06:16:27.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.581+0000 7f4a75c9d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4a70109470 con 0x7f4a70101a60 2024-09-19T06:16:27.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.581+0000 7f4a6f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a70100860 0x7f4a7006a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:27.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.581+0000 7f4a6f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a70100860 0x7f4a7006a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51838/0 (socket says 172.21.15.53:51838) 2024-09-19T06:16:27.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.581+0000 7f4a6f7fe640 1 -- 172.21.15.53:0/1918324 learned_addr learned my addr 172.21.15.53:0/1918324 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:27.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.581+0000 7f4a6effd640 1 --2- 172.21.15.53:0/1918324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a70101a60 0x7f4a7019c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:27.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.581+0000 7f4a6f7fe640 1 -- 172.21.15.53:0/1918324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a70101a60 msgr2=0x7f4a7019c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:27.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.581+0000 7f4a6f7fe640 1 --2- 172.21.15.53:0/1918324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a70101a60 0x7f4a7019c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:27.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.581+0000 7f4a6f7fe640 1 -- 172.21.15.53:0/1918324 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a7019c670 con 0x7f4a70100860 2024-09-19T06:16:27.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.581+0000 7f4a6effd640 1 --2- 172.21.15.53:0/1918324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a70101a60 0x7f4a7019c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:16:27.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.582+0000 7f4a6f7fe640 1 --2- 172.21.15.53:0/1918324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a70100860 0x7f4a7006a790 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000d720 tx=0x7f4a6000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:27.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.582+0000 7f4a6cff9640 1 -- 172.21.15.53:0/1918324 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4a60017070 con 0x7f4a70100860 2024-09-19T06:16:27.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.582+0000 7f4a6cff9640 1 -- 172.21.15.53:0/1918324 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a60009820 con 0x7f4a70100860 2024-09-19T06:16:27.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.582+0000 7f4a75c9d640 1 -- 172.21.15.53:0/1918324 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a7019c960 con 0x7f4a70100860 2024-09-19T06:16:27.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.582+0000 7f4a6cff9640 1 -- 172.21.15.53:0/1918324 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4a600039f0 con 0x7f4a70100860 2024-09-19T06:16:27.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.582+0000 7f4a75c9d640 1 -- 172.21.15.53:0/1918324 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a7019cf70 con 0x7f4a70100860 2024-09-19T06:16:27.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.583+0000 7f4a75c9d640 1 -- 172.21.15.53:0/1918324 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a70100c60 con 0x7f4a70100860 2024-09-19T06:16:27.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.583+0000 7f4a6cff9640 1 -- 172.21.15.53:0/1918324 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f4a600099d0 con 0x7f4a70100860 2024-09-19T06:16:27.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.584+0000 7f4a6cff9640 1 --2- 172.21.15.53:0/1918324 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f4a5806e2b0 0x7f4a58070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:27.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.584+0000 7f4a6cff9640 1 -- 172.21.15.53:0/1918324 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5502+0+0 (secure 0 0 0) 0x7f4a6008dde0 con 0x7f4a70100860 2024-09-19T06:16:27.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.584+0000 7f4a6effd640 1 --2- 172.21.15.53:0/1918324 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f4a5806e2b0 0x7f4a58070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:27.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.585+0000 7f4a6effd640 1 --2- 172.21.15.53:0/1918324 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f4a5806e2b0 0x7f4a58070770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4a7006a510 tx=0x7f4a64004e20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:27.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.588+0000 7f4a6cff9640 1 -- 172.21.15.53:0/1918324 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4a6005b690 con 0x7f4a70100860 2024-09-19T06:16:27.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.768+0000 7f4a75c9d640 1 -- 172.21.15.53:0/1918324 --> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f4a70062630 con 0x7f4a5806e2b0 2024-09-19T06:16:27.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.769+0000 7f4a6cff9640 1 -- 172.21.15.53:0/1918324 <== mgr.14217 v2:172.21.15.53:6800/3884355752 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+148 (secure 0 0 0) 0x7f4a70062630 con 0x7f4a5806e2b0 2024-09-19T06:16:27.771 INFO:teuthology.orchestra.run.smithi053.stdout:HOST ADDR LABELS STATUS 2024-09-19T06:16:27.771 INFO:teuthology.orchestra.run.smithi053.stdout:smithi053 172.21.15.53 2024-09-19T06:16:27.771 INFO:teuthology.orchestra.run.smithi053.stdout:smithi155 172.21.15.155 2024-09-19T06:16:27.771 INFO:teuthology.orchestra.run.smithi053.stdout:2 hosts in cluster 2024-09-19T06:16:27.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.771+0000 7f4a75c9d640 1 -- 172.21.15.53:0/1918324 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f4a5806e2b0 msgr2=0x7f4a58070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:27.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.771+0000 7f4a75c9d640 1 --2- 172.21.15.53:0/1918324 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f4a5806e2b0 0x7f4a58070770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4a7006a510 tx=0x7f4a64004e20 comp rx=0 tx=0).stop 2024-09-19T06:16:27.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.771+0000 7f4a75c9d640 1 -- 172.21.15.53:0/1918324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a70100860 msgr2=0x7f4a7006a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:27.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.771+0000 7f4a75c9d640 1 --2- 172.21.15.53:0/1918324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a70100860 0x7f4a7006a790 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000d720 tx=0x7f4a6000dbe0 comp rx=0 tx=0).stop 2024-09-19T06:16:27.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.771+0000 7f4a75c9d640 1 -- 172.21.15.53:0/1918324 shutdown_connections 2024-09-19T06:16:27.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.771+0000 7f4a75c9d640 1 --2- 172.21.15.53:0/1918324 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f4a5806e2b0 0x7f4a58070770 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:27.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.771+0000 7f4a75c9d640 1 --2- 172.21.15.53:0/1918324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a70101a60 0x7f4a7019c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:27.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.771+0000 7f4a75c9d640 1 --2- 172.21.15.53:0/1918324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a70100860 0x7f4a7006a790 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:27.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.771+0000 7f4a75c9d640 1 -- 172.21.15.53:0/1918324 >> 172.21.15.53:0/1918324 conn(0x7f4a700fc010 msgr2=0x7f4a70104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:27.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.771+0000 7f4a75c9d640 1 -- 172.21.15.53:0/1918324 shutdown_connections 2024-09-19T06:16:27.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:27.771+0000 7f4a75c9d640 1 -- 172.21.15.53:0/1918324 wait complete. 2024-09-19T06:16:28.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:27 smithi053 ceph-mon[27968]: from='client.14494 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:16:28.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:27 smithi155 ceph-mon[35292]: from='client.14494 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:16:28.438 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-19T06:16:28.696 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:28.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:28 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:16:28.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:28 smithi155 ceph-mon[35292]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:29.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:28 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:16:29.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:28 smithi053 ceph-mon[27968]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:29.141 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.137+0000 7f0ab5e35640 1 -- 172.21.15.53:0/1009213656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ab0101960 msgr2=0x7f0ab0101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:29.141 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.137+0000 7f0ab5e35640 1 --2- 172.21.15.53:0/1009213656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ab0101960 0x7f0ab0101d60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0009880 tx=0x7f0aa002a4a0 comp rx=0 tx=0).stop 2024-09-19T06:16:29.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.137+0000 7f0ab5e35640 1 -- 172.21.15.53:0/1009213656 shutdown_connections 2024-09-19T06:16:29.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.137+0000 7f0ab5e35640 1 --2- 172.21.15.53:0/1009213656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ab0102b60 0x7f0ab0102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:29.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.137+0000 7f0ab5e35640 1 --2- 172.21.15.53:0/1009213656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ab0101960 0x7f0ab0101d60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:29.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.137+0000 7f0ab5e35640 1 -- 172.21.15.53:0/1009213656 >> 172.21.15.53:0/1009213656 conn(0x7f0ab00fd0f0 msgr2=0x7f0ab00ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:29.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.137+0000 7f0ab5e35640 1 -- 172.21.15.53:0/1009213656 shutdown_connections 2024-09-19T06:16:29.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.137+0000 7f0ab5e35640 1 -- 172.21.15.53:0/1009213656 wait complete. 2024-09-19T06:16:29.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.138+0000 7f0ab5e35640 1 Processor -- start 2024-09-19T06:16:29.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.138+0000 7f0ab5e35640 1 -- start start 2024-09-19T06:16:29.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.138+0000 7f0ab5e35640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ab0101960 0x7f0ab0199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:29.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.138+0000 7f0ab5e35640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ab0102b60 0x7f0ab0199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:29.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.138+0000 7f0ab5e35640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab010a770 con 0x7f0ab0102b60 2024-09-19T06:16:29.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.138+0000 7f0ab5e35640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab010a5f0 con 0x7f0ab0101960 2024-09-19T06:16:29.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.138+0000 7f0aaeffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ab0102b60 0x7f0ab0199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:29.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.138+0000 7f0aaeffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ab0102b60 0x7f0ab0199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51854/0 (socket says 172.21.15.53:51854) 2024-09-19T06:16:29.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.138+0000 7f0aaeffd640 1 -- 172.21.15.53:0/4091084199 learned_addr learned my addr 172.21.15.53:0/4091084199 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:29.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.138+0000 7f0aaf7fe640 1 --2- 172.21.15.53:0/4091084199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ab0101960 0x7f0ab0199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:29.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.139+0000 7f0aaeffd640 1 -- 172.21.15.53:0/4091084199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ab0101960 msgr2=0x7f0ab0199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:29.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.139+0000 7f0aaeffd640 1 --2- 172.21.15.53:0/4091084199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ab0101960 0x7f0ab0199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:29.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.139+0000 7f0aaeffd640 1 -- 172.21.15.53:0/4091084199 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ab019a250 con 0x7f0ab0102b60 2024-09-19T06:16:29.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.139+0000 7f0aaeffd640 1 --2- 172.21.15.53:0/4091084199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ab0102b60 0x7f0ab0199c60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0aa400b420 tx=0x7f0aa400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:29.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.139+0000 7f0aacff9640 1 -- 172.21.15.53:0/4091084199 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0aa4011630 con 0x7f0ab0102b60 2024-09-19T06:16:29.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.139+0000 7f0aacff9640 1 -- 172.21.15.53:0/4091084199 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0aa4016070 con 0x7f0ab0102b60 2024-09-19T06:16:29.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.139+0000 7f0ab5e35640 1 -- 172.21.15.53:0/4091084199 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ab019ecc0 con 0x7f0ab0102b60 2024-09-19T06:16:29.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.139+0000 7f0aacff9640 1 -- 172.21.15.53:0/4091084199 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0aa4010c90 con 0x7f0ab0102b60 2024-09-19T06:16:29.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.139+0000 7f0ab5e35640 1 -- 172.21.15.53:0/4091084199 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ab019f2a0 con 0x7f0ab0102b60 2024-09-19T06:16:29.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.140+0000 7f0ab5e35640 1 -- 172.21.15.53:0/4091084199 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ab0101de0 con 0x7f0ab0102b60 2024-09-19T06:16:29.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.141+0000 7f0aacff9640 1 -- 172.21.15.53:0/4091084199 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f0aa40117d0 con 0x7f0ab0102b60 2024-09-19T06:16:29.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.141+0000 7f0aacff9640 1 --2- 172.21.15.53:0/4091084199 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f0a9806e2b0 0x7f0a98070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:29.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.141+0000 7f0aacff9640 1 -- 172.21.15.53:0/4091084199 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5502+0+0 (secure 0 0 0) 0x7f0aa408d9e0 con 0x7f0ab0102b60 2024-09-19T06:16:29.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.141+0000 7f0aaf7fe640 1 --2- 172.21.15.53:0/4091084199 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f0a9806e2b0 0x7f0a98070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:29.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.142+0000 7f0aaf7fe640 1 --2- 172.21.15.53:0/4091084199 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f0a9806e2b0 0x7f0a98070770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0000c00 tx=0x7f0aa0004e20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:29.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.146+0000 7f0aacff9640 1 -- 172.21.15.53:0/4091084199 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0aa405b290 con 0x7f0ab0102b60 2024-09-19T06:16:29.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.329+0000 7f0ab5e35640 1 -- 172.21.15.53:0/4091084199 --> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f0ab002c640 con 0x7f0a9806e2b0 2024-09-19T06:16:29.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.330+0000 7f0aacff9640 1 -- 172.21.15.53:0/4091084199 <== mgr.14217 v2:172.21.15.53:6800/3884355752 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+399 (secure 0 0 0) 0x7f0ab002c640 con 0x7f0a9806e2b0 2024-09-19T06:16:29.332 INFO:teuthology.orchestra.run.smithi053.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-19T06:16:29.332 INFO:teuthology.orchestra.run.smithi053.stdout:smithi053 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300G8480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-09-19T06:16:29.332 INFO:teuthology.orchestra.run.smithi053.stdout:smithi155 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GW400BGN 372G No 41s ago Has a FileSystem, LVM detected 2024-09-19T06:16:29.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.333+0000 7f0ab5e35640 1 -- 172.21.15.53:0/4091084199 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f0a9806e2b0 msgr2=0x7f0a98070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:29.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.333+0000 7f0ab5e35640 1 --2- 172.21.15.53:0/4091084199 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f0a9806e2b0 0x7f0a98070770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0000c00 tx=0x7f0aa0004e20 comp rx=0 tx=0).stop 2024-09-19T06:16:29.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.333+0000 7f0ab5e35640 1 -- 172.21.15.53:0/4091084199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ab0102b60 msgr2=0x7f0ab0199c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:29.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.333+0000 7f0ab5e35640 1 --2- 172.21.15.53:0/4091084199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ab0102b60 0x7f0ab0199c60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0aa400b420 tx=0x7f0aa400b8e0 comp rx=0 tx=0).stop 2024-09-19T06:16:29.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.333+0000 7f0ab5e35640 1 -- 172.21.15.53:0/4091084199 shutdown_connections 2024-09-19T06:16:29.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.333+0000 7f0ab5e35640 1 --2- 172.21.15.53:0/4091084199 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f0a9806e2b0 0x7f0a98070770 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:29.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.333+0000 7f0ab5e35640 1 --2- 172.21.15.53:0/4091084199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ab0102b60 0x7f0ab0199c60 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:29.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.333+0000 7f0ab5e35640 1 --2- 172.21.15.53:0/4091084199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ab0101960 0x7f0ab0199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:29.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.333+0000 7f0ab5e35640 1 -- 172.21.15.53:0/4091084199 >> 172.21.15.53:0/4091084199 conn(0x7f0ab00fd0f0 msgr2=0x7f0ab00747a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:29.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.333+0000 7f0ab5e35640 1 -- 172.21.15.53:0/4091084199 shutdown_connections 2024-09-19T06:16:29.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:29.333+0000 7f0ab5e35640 1 -- 172.21.15.53:0/4091084199 wait complete. 2024-09-19T06:16:29.977 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:29 smithi053 ceph-mon[27968]: from='client.14498 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:16:30.007 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T06:16:30.017 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi053.front.sepia.ceph.com 2024-09-19T06:16:30.018 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- bash -c 'ceph fs volume create foo' 2024-09-19T06:16:30.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:29 smithi155 ceph-mon[35292]: from='client.14498 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:16:30.277 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:30.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.713+0000 7f628713f640 1 -- 172.21.15.53:0/1035570025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62801023f0 msgr2=0x7f62801047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:30.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.713+0000 7f628713f640 1 --2- 172.21.15.53:0/1035570025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62801023f0 0x7f62801047e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f62700098b0 tx=0x7f627002a4d0 comp rx=0 tx=0).stop 2024-09-19T06:16:30.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.713+0000 7f628713f640 1 -- 172.21.15.53:0/1035570025 shutdown_connections 2024-09-19T06:16:30.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.713+0000 7f628713f640 1 --2- 172.21.15.53:0/1035570025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6280104d20 0x7f6280107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:30.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.713+0000 7f628713f640 1 --2- 172.21.15.53:0/1035570025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62801023f0 0x7f62801047e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:30.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.713+0000 7f628713f640 1 -- 172.21.15.53:0/1035570025 >> 172.21.15.53:0/1035570025 conn(0x7f62800fc010 msgr2=0x7f62800fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:30.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.714+0000 7f628713f640 1 -- 172.21.15.53:0/1035570025 shutdown_connections 2024-09-19T06:16:30.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.714+0000 7f628713f640 1 -- 172.21.15.53:0/1035570025 wait complete. 2024-09-19T06:16:30.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.714+0000 7f628713f640 1 Processor -- start 2024-09-19T06:16:30.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.714+0000 7f628713f640 1 -- start start 2024-09-19T06:16:30.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.714+0000 7f628713f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62801023f0 0x7f6280101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:30.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.714+0000 7f628713f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6280104d20 0x7f62801003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:30.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.714+0000 7f628713f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f628010a7a0 con 0x7f62801023f0 2024-09-19T06:16:30.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.714+0000 7f628713f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f628010a620 con 0x7f6280104d20 2024-09-19T06:16:30.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.715+0000 7f6284edb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62801023f0 0x7f6280101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:30.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.715+0000 7f6284edb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62801023f0 0x7f6280101ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51874/0 (socket says 172.21.15.53:51874) 2024-09-19T06:16:30.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.715+0000 7f6284edb640 1 -- 172.21.15.53:0/1278173077 learned_addr learned my addr 172.21.15.53:0/1278173077 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:30.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.715+0000 7f6284edb640 1 -- 172.21.15.53:0/1278173077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6280104d20 msgr2=0x7f62801003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:30.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.715+0000 7f6284edb640 1 --2- 172.21.15.53:0/1278173077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6280104d20 0x7f62801003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:30.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.715+0000 7f6284edb640 1 -- 172.21.15.53:0/1278173077 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62801008e0 con 0x7f62801023f0 2024-09-19T06:16:30.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.715+0000 7f6284edb640 1 --2- 172.21.15.53:0/1278173077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62801023f0 0x7f6280101ce0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6270009230 tx=0x7f62700048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:30.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.715+0000 7f627dffb640 1 -- 172.21.15.53:0/1278173077 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f627003f070 con 0x7f62801023f0 2024-09-19T06:16:30.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.715+0000 7f627dffb640 1 -- 172.21.15.53:0/1278173077 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6270004480 con 0x7f62801023f0 2024-09-19T06:16:30.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.715+0000 7f628713f640 1 -- 172.21.15.53:0/1278173077 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6280100b70 con 0x7f62801023f0 2024-09-19T06:16:30.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.715+0000 7f627dffb640 1 -- 172.21.15.53:0/1278173077 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6270035030 con 0x7f62801023f0 2024-09-19T06:16:30.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.716+0000 7f628713f640 1 -- 172.21.15.53:0/1278173077 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6280101120 con 0x7f62801023f0 2024-09-19T06:16:30.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.717+0000 7f628713f640 1 -- 172.21.15.53:0/1278173077 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6280102850 con 0x7f62801023f0 2024-09-19T06:16:30.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.717+0000 7f627dffb640 1 -- 172.21.15.53:0/1278173077 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f6270042070 con 0x7f62801023f0 2024-09-19T06:16:30.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.718+0000 7f627dffb640 1 --2- 172.21.15.53:0/1278173077 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f626806e310 0x7f62680707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:30.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.718+0000 7f627dffb640 1 -- 172.21.15.53:0/1278173077 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5502+0+0 (secure 0 0 0) 0x7f62700aede0 con 0x7f62801023f0 2024-09-19T06:16:30.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.718+0000 7f627ffff640 1 --2- 172.21.15.53:0/1278173077 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f626806e310 0x7f62680707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:30.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.718+0000 7f627ffff640 1 --2- 172.21.15.53:0/1278173077 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f626806e310 0x7f62680707d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6280101a60 tx=0x7f6274006c90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:30.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.722+0000 7f627dffb640 1 -- 172.21.15.53:0/1278173077 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f627007c710 con 0x7f62801023f0 2024-09-19T06:16:30.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:30.927+0000 7f628713f640 1 -- 172.21.15.53:0/1278173077 --> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "foo", "target": ["mon-mgr", ""]}) -- 0x7f62801055e0 con 0x7f626806e310 2024-09-19T06:16:31.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:30 smithi053 ceph-mon[27968]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:31.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:30 smithi155 ceph-mon[35292]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:32.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:31 smithi155 ceph-mon[35292]: from='client.14502 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "foo", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:16:32.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:31 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd pool create", "pool": "cephfs.foo.meta"}]: dispatch 2024-09-19T06:16:32.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:31 smithi053 ceph-mon[27968]: from='client.14502 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "foo", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:16:32.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:31 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd pool create", "pool": "cephfs.foo.meta"}]: dispatch 2024-09-19T06:16:32.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:32.902+0000 7f627dffb640 1 -- 172.21.15.53:0/1278173077 <== mgr.14217 v2:172.21.15.53:6800/3884355752 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f62801055e0 con 0x7f626806e310 2024-09-19T06:16:32.905 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:32 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "osd pool create", "pool": "cephfs.foo.meta"}]': finished 2024-09-19T06:16:32.905 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:32 smithi053 ceph-mon[27968]: osdmap e61: 8 total, 8 up, 8 in 2024-09-19T06:16:32.905 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:32 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.foo.data"}]: dispatch 2024-09-19T06:16:32.905 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:32 smithi053 ceph-mon[27968]: pgmap v201: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:32.906 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:32 smithi053 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-mon-smithi053[27964]: 2024-09-19T06:16:32.877+0000 7f07135d7640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T06:16:32.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:32.905+0000 7f628713f640 1 -- 172.21.15.53:0/1278173077 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f626806e310 msgr2=0x7f62680707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:32.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:32.905+0000 7f628713f640 1 --2- 172.21.15.53:0/1278173077 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f626806e310 0x7f62680707d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6280101a60 tx=0x7f6274006c90 comp rx=0 tx=0).stop 2024-09-19T06:16:32.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:32.905+0000 7f628713f640 1 -- 172.21.15.53:0/1278173077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62801023f0 msgr2=0x7f6280101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:32.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:32.905+0000 7f628713f640 1 --2- 172.21.15.53:0/1278173077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62801023f0 0x7f6280101ce0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6270009230 tx=0x7f62700048f0 comp rx=0 tx=0).stop 2024-09-19T06:16:32.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:32.905+0000 7f628713f640 1 -- 172.21.15.53:0/1278173077 shutdown_connections 2024-09-19T06:16:32.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:32.905+0000 7f628713f640 1 --2- 172.21.15.53:0/1278173077 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f626806e310 0x7f62680707d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:32.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:32.905+0000 7f628713f640 1 --2- 172.21.15.53:0/1278173077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6280104d20 0x7f62801003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:32.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:32.905+0000 7f628713f640 1 --2- 172.21.15.53:0/1278173077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62801023f0 0x7f6280101ce0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:32.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:32.905+0000 7f628713f640 1 -- 172.21.15.53:0/1278173077 >> 172.21.15.53:0/1278173077 conn(0x7f62800fc010 msgr2=0x7f62800fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:32.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:32.905+0000 7f628713f640 1 -- 172.21.15.53:0/1278173077 shutdown_connections 2024-09-19T06:16:32.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:32.905+0000 7f628713f640 1 -- 172.21.15.53:0/1278173077 wait complete. 2024-09-19T06:16:33.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:32 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "osd pool create", "pool": "cephfs.foo.meta"}]': finished 2024-09-19T06:16:33.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:32 smithi155 ceph-mon[35292]: osdmap e61: 8 total, 8 up, 8 in 2024-09-19T06:16:33.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:32 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.foo.data"}]: dispatch 2024-09-19T06:16:33.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:32 smithi155 ceph-mon[35292]: pgmap v201: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:33.579 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- bash -c 'ceph fs volume create bar' 2024-09-19T06:16:33.853 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:33.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:33 smithi155 ceph-mon[35292]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T06:16:33.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:33 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.foo.data"}]': finished 2024-09-19T06:16:33.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:33 smithi155 ceph-mon[35292]: osdmap e62: 8 total, 8 up, 8 in 2024-09-19T06:16:33.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:33 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "fs new", "fs_name": "foo", "metadata": "cephfs.foo.meta", "data": "cephfs.foo.data"}]: dispatch 2024-09-19T06:16:33.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:33 smithi155 ceph-mon[35292]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T06:16:33.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:33 smithi155 ceph-mon[35292]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T06:16:33.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:33 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "fs new", "fs_name": "foo", "metadata": "cephfs.foo.meta", "data": "cephfs.foo.data"}]': finished 2024-09-19T06:16:33.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:33 smithi155 ceph-mon[35292]: osdmap e63: 8 total, 8 up, 8 in 2024-09-19T06:16:33.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:33 smithi155 ceph-mon[35292]: fsmap foo:0 2024-09-19T06:16:33.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:33 smithi155 ceph-mon[35292]: Saving service mds.foo spec with placement count:2 2024-09-19T06:16:33.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:33 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:33.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:33 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:16:33.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:33 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:16:33.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:33 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:16:33.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:33 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:33.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:33 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.foo.smithi053.rdifzk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:16:33.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:33 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.foo.smithi053.rdifzk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:16:33.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:33 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:16:33.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:33 smithi155 ceph-mon[35292]: Deploying daemon mds.foo.smithi053.rdifzk on smithi053 2024-09-19T06:16:34.129 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:33 smithi053 ceph-mon[27968]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T06:16:34.129 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:33 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.foo.data"}]': finished 2024-09-19T06:16:34.130 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:33 smithi053 ceph-mon[27968]: osdmap e62: 8 total, 8 up, 8 in 2024-09-19T06:16:34.130 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:33 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "fs new", "fs_name": "foo", "metadata": "cephfs.foo.meta", "data": "cephfs.foo.data"}]: dispatch 2024-09-19T06:16:34.130 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:33 smithi053 ceph-mon[27968]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T06:16:34.130 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:33 smithi053 ceph-mon[27968]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T06:16:34.130 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:33 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "fs new", "fs_name": "foo", "metadata": "cephfs.foo.meta", "data": "cephfs.foo.data"}]': finished 2024-09-19T06:16:34.130 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:33 smithi053 ceph-mon[27968]: osdmap e63: 8 total, 8 up, 8 in 2024-09-19T06:16:34.130 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:33 smithi053 ceph-mon[27968]: fsmap foo:0 2024-09-19T06:16:34.130 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:33 smithi053 ceph-mon[27968]: Saving service mds.foo spec with placement count:2 2024-09-19T06:16:34.130 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:33 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:34.130 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:33 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:16:34.130 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:33 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:16:34.130 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:33 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:16:34.130 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:33 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:34.130 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:33 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.foo.smithi053.rdifzk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:16:34.131 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:33 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.foo.smithi053.rdifzk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:16:34.131 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:33 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:16:34.131 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:33 smithi053 ceph-mon[27968]: Deploying daemon mds.foo.smithi053.rdifzk on smithi053 2024-09-19T06:16:34.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.330+0000 7ff462bd0640 1 -- 172.21.15.53:0/40086595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff45c102b60 msgr2=0x7ff45c102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:34.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.330+0000 7ff462bd0640 1 --2- 172.21.15.53:0/40086595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff45c102b60 0x7ff45c102fe0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff448009880 tx=0x7ff44802a4a0 comp rx=0 tx=0).stop 2024-09-19T06:16:34.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.330+0000 7ff462bd0640 1 -- 172.21.15.53:0/40086595 shutdown_connections 2024-09-19T06:16:34.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.330+0000 7ff462bd0640 1 --2- 172.21.15.53:0/40086595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff45c102b60 0x7ff45c102fe0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:34.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.330+0000 7ff462bd0640 1 --2- 172.21.15.53:0/40086595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff45c101960 0x7ff45c101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:34.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.330+0000 7ff462bd0640 1 -- 172.21.15.53:0/40086595 >> 172.21.15.53:0/40086595 conn(0x7ff45c0fd0f0 msgr2=0x7ff45c0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:34.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.330+0000 7ff462bd0640 1 -- 172.21.15.53:0/40086595 shutdown_connections 2024-09-19T06:16:34.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.330+0000 7ff462bd0640 1 -- 172.21.15.53:0/40086595 wait complete. 2024-09-19T06:16:34.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.331+0000 7ff462bd0640 1 Processor -- start 2024-09-19T06:16:34.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.331+0000 7ff462bd0640 1 -- start start 2024-09-19T06:16:34.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.331+0000 7ff462bd0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff45c101960 0x7ff45c197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:34.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.331+0000 7ff462bd0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff45c102b60 0x7ff45c197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:34.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.331+0000 7ff462bd0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c10a790 con 0x7ff45c102b60 2024-09-19T06:16:34.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.331+0000 7ff462bd0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c10a610 con 0x7ff45c101960 2024-09-19T06:16:34.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.331+0000 7ff45bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff45c102b60 0x7ff45c197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:34.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.331+0000 7ff45bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff45c102b60 0x7ff45c197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59926/0 (socket says 172.21.15.53:59926) 2024-09-19T06:16:34.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.331+0000 7ff45bfff640 1 -- 172.21.15.53:0/3905190248 learned_addr learned my addr 172.21.15.53:0/3905190248 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:34.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.331+0000 7ff46096c640 1 --2- 172.21.15.53:0/3905190248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff45c101960 0x7ff45c197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:34.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.332+0000 7ff45bfff640 1 -- 172.21.15.53:0/3905190248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff45c101960 msgr2=0x7ff45c197520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:34.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.332+0000 7ff45bfff640 1 --2- 172.21.15.53:0/3905190248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff45c101960 0x7ff45c197520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:34.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.332+0000 7ff45bfff640 1 -- 172.21.15.53:0/3905190248 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45c198050 con 0x7ff45c102b60 2024-09-19T06:16:34.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.332+0000 7ff45bfff640 1 --2- 172.21.15.53:0/3905190248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff45c102b60 0x7ff45c197a60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff448004e80 tx=0x7ff4480048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:34.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.332+0000 7ff459ffb640 1 -- 172.21.15.53:0/3905190248 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff44803f070 con 0x7ff45c102b60 2024-09-19T06:16:34.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.332+0000 7ff462bd0640 1 -- 172.21.15.53:0/3905190248 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff45c19ca60 con 0x7ff45c102b60 2024-09-19T06:16:34.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.332+0000 7ff459ffb640 1 -- 172.21.15.53:0/3905190248 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4480044d0 con 0x7ff45c102b60 2024-09-19T06:16:34.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.332+0000 7ff459ffb640 1 -- 172.21.15.53:0/3905190248 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff448035030 con 0x7ff45c102b60 2024-09-19T06:16:34.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.332+0000 7ff462bd0640 1 -- 172.21.15.53:0/3905190248 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff45c19cfe0 con 0x7ff45c102b60 2024-09-19T06:16:34.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.333+0000 7ff462bd0640 1 -- 172.21.15.53:0/3905190248 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff45c101de0 con 0x7ff45c102b60 2024-09-19T06:16:34.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.334+0000 7ff459ffb640 1 -- 172.21.15.53:0/3905190248 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7ff448042070 con 0x7ff45c102b60 2024-09-19T06:16:34.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.334+0000 7ff459ffb640 1 --2- 172.21.15.53:0/3905190248 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7ff44c06e2b0 0x7ff44c070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:34.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.334+0000 7ff459ffb640 1 -- 172.21.15.53:0/3905190248 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6320+0+0 (secure 0 0 0) 0x7ff4480af1e0 con 0x7ff45c102b60 2024-09-19T06:16:34.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.334+0000 7ff46096c640 1 --2- 172.21.15.53:0/3905190248 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7ff44c06e2b0 0x7ff44c070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:34.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.335+0000 7ff46096c640 1 --2- 172.21.15.53:0/3905190248 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7ff44c06e2b0 0x7ff44c070770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff45c1029c0 tx=0x7ff450008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:34.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.339+0000 7ff459ffb640 1 -- 172.21.15.53:0/3905190248 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff44807c760 con 0x7ff45c102b60 2024-09-19T06:16:34.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:34.541+0000 7ff462bd0640 1 -- 172.21.15.53:0/3905190248 --> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "bar", "target": ["mon-mgr", ""]}) -- 0x7ff45c02c640 con 0x7ff44c06e2b0 2024-09-19T06:16:35.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:34 smithi053 ceph-mon[27968]: osdmap e64: 8 total, 8 up, 8 in 2024-09-19T06:16:35.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:34 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd pool create", "pool": "cephfs.bar.meta"}]: dispatch 2024-09-19T06:16:35.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:34 smithi155 ceph-mon[35292]: osdmap e64: 8 total, 8 up, 8 in 2024-09-19T06:16:35.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:34 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd pool create", "pool": "cephfs.bar.meta"}]: dispatch 2024-09-19T06:16:36.296 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:35 smithi053 ceph-mon[27968]: from='client.14506 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "bar", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:16:36.296 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:35 smithi053 ceph-mon[27968]: pgmap v205: 65 pgs: 20 active+clean, 21 creating+peering, 24 unknown; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:36.296 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:35 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "osd pool create", "pool": "cephfs.bar.meta"}]': finished 2024-09-19T06:16:36.296 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:35 smithi053 ceph-mon[27968]: osdmap e65: 8 total, 8 up, 8 in 2024-09-19T06:16:36.296 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:35 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.bar.data"}]: dispatch 2024-09-19T06:16:36.296 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:35 smithi053 ceph-mon[27968]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-19T06:16:36.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:35 smithi155 ceph-mon[35292]: from='client.14506 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "bar", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:16:36.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:35 smithi155 ceph-mon[35292]: pgmap v205: 65 pgs: 20 active+clean, 21 creating+peering, 24 unknown; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:36.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:35 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "osd pool create", "pool": "cephfs.bar.meta"}]': finished 2024-09-19T06:16:36.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:35 smithi155 ceph-mon[35292]: osdmap e65: 8 total, 8 up, 8 in 2024-09-19T06:16:36.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:35 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.bar.data"}]: dispatch 2024-09-19T06:16:36.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:35 smithi155 ceph-mon[35292]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-19T06:16:36.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:36.807+0000 7ff459ffb640 1 -- 172.21.15.53:0/3905190248 <== mgr.14217 v2:172.21.15.53:6800/3884355752 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff45c02c640 con 0x7ff44c06e2b0 2024-09-19T06:16:36.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:36.811+0000 7ff462bd0640 1 -- 172.21.15.53:0/3905190248 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7ff44c06e2b0 msgr2=0x7ff44c070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:36.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:36.811+0000 7ff462bd0640 1 --2- 172.21.15.53:0/3905190248 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7ff44c06e2b0 0x7ff44c070770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff45c1029c0 tx=0x7ff450008040 comp rx=0 tx=0).stop 2024-09-19T06:16:36.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:36.811+0000 7ff462bd0640 1 -- 172.21.15.53:0/3905190248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff45c102b60 msgr2=0x7ff45c197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:36.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:36.811+0000 7ff462bd0640 1 --2- 172.21.15.53:0/3905190248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff45c102b60 0x7ff45c197a60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff448004e80 tx=0x7ff4480048f0 comp rx=0 tx=0).stop 2024-09-19T06:16:36.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:36.812+0000 7ff462bd0640 1 -- 172.21.15.53:0/3905190248 shutdown_connections 2024-09-19T06:16:36.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:36.812+0000 7ff462bd0640 1 --2- 172.21.15.53:0/3905190248 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7ff44c06e2b0 0x7ff44c070770 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:36.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:36.812+0000 7ff462bd0640 1 --2- 172.21.15.53:0/3905190248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff45c102b60 0x7ff45c197a60 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:36.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:36.812+0000 7ff462bd0640 1 --2- 172.21.15.53:0/3905190248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff45c101960 0x7ff45c197520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:36.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:36.812+0000 7ff462bd0640 1 -- 172.21.15.53:0/3905190248 >> 172.21.15.53:0/3905190248 conn(0x7ff45c0fd0f0 msgr2=0x7ff45c106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:36.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:36.812+0000 7ff462bd0640 1 -- 172.21.15.53:0/3905190248 shutdown_connections 2024-09-19T06:16:36.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:36.812+0000 7ff462bd0640 1 -- 172.21.15.53:0/3905190248 wait complete. 2024-09-19T06:16:37.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:36 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.bar.data"}]': finished 2024-09-19T06:16:37.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:36 smithi053 ceph-mon[27968]: osdmap e66: 8 total, 8 up, 8 in 2024-09-19T06:16:37.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:36 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "fs new", "fs_name": "bar", "metadata": "cephfs.bar.meta", "data": "cephfs.bar.data"}]: dispatch 2024-09-19T06:16:37.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:36 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:37.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:36 smithi053 ceph-mon[27968]: osdmap e67: 8 total, 8 up, 8 in 2024-09-19T06:16:37.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:36 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:37.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:36 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:37.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:36 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.foo.smithi155.dsbavw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:16:37.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:36 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.foo.smithi155.dsbavw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:16:37.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:36 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:16:37.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:36 smithi053 ceph-mon[27968]: Deploying daemon mds.foo.smithi155.dsbavw on smithi155 2024-09-19T06:16:37.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:36 smithi053 ceph-mon[27968]: pgmap v209: 129 pgs: 45 active+clean, 29 creating+peering, 55 unknown; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:37.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:36 smithi053 ceph-mon[27968]: daemon mds.foo.smithi053.rdifzk assigned to filesystem foo as rank 0 (now has 1 ranks) 2024-09-19T06:16:37.107 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:36 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "fs new", "fs_name": "bar", "metadata": "cephfs.bar.meta", "data": "cephfs.bar.data"}]': finished 2024-09-19T06:16:37.107 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:36 smithi053 ceph-mon[27968]: mds.? [v2:172.21.15.53:6834/2573849324,v1:172.21.15.53:6835/2573849324] up:boot 2024-09-19T06:16:37.107 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:36 smithi053 ceph-mon[27968]: fsmap foo:1 bar:0 {foo:0=foo.smithi053.rdifzk=up:creating} 2024-09-19T06:16:37.107 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:36 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mds metadata", "who": "foo.smithi053.rdifzk"}]: dispatch 2024-09-19T06:16:37.107 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:36 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:37.107 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:36 smithi053 ceph-mon[27968]: daemon mds.foo.smithi053.rdifzk is now active in filesystem foo as rank 0 2024-09-19T06:16:37.164 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:36 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.bar.data"}]': finished 2024-09-19T06:16:37.164 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:36 smithi155 ceph-mon[35292]: osdmap e66: 8 total, 8 up, 8 in 2024-09-19T06:16:37.164 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:36 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "fs new", "fs_name": "bar", "metadata": "cephfs.bar.meta", "data": "cephfs.bar.data"}]: dispatch 2024-09-19T06:16:37.164 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:36 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:37.164 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:36 smithi155 ceph-mon[35292]: osdmap e67: 8 total, 8 up, 8 in 2024-09-19T06:16:37.164 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:36 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:37.164 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:36 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:37.164 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:36 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.foo.smithi155.dsbavw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:16:37.164 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:36 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.foo.smithi155.dsbavw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:16:37.165 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:36 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:16:37.165 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:36 smithi155 ceph-mon[35292]: Deploying daemon mds.foo.smithi155.dsbavw on smithi155 2024-09-19T06:16:37.165 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:36 smithi155 ceph-mon[35292]: pgmap v209: 129 pgs: 45 active+clean, 29 creating+peering, 55 unknown; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:16:37.165 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:36 smithi155 ceph-mon[35292]: daemon mds.foo.smithi053.rdifzk assigned to filesystem foo as rank 0 (now has 1 ranks) 2024-09-19T06:16:37.165 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:36 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "fs new", "fs_name": "bar", "metadata": "cephfs.bar.meta", "data": "cephfs.bar.data"}]': finished 2024-09-19T06:16:37.165 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:36 smithi155 ceph-mon[35292]: mds.? [v2:172.21.15.53:6834/2573849324,v1:172.21.15.53:6835/2573849324] up:boot 2024-09-19T06:16:37.165 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:36 smithi155 ceph-mon[35292]: fsmap foo:1 bar:0 {foo:0=foo.smithi053.rdifzk=up:creating} 2024-09-19T06:16:37.165 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:36 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mds metadata", "who": "foo.smithi053.rdifzk"}]: dispatch 2024-09-19T06:16:37.165 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:36 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:37.165 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:36 smithi155 ceph-mon[35292]: daemon mds.foo.smithi053.rdifzk is now active in filesystem foo as rank 0 2024-09-19T06:16:37.582 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-19T06:16:37.600 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:16:37.867 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:38.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.326+0000 7f0b83f84640 1 -- 172.21.15.53:0/567823801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b7c102ab0 msgr2=0x7f0b7c102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:38.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.326+0000 7f0b83f84640 1 --2- 172.21.15.53:0/567823801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b7c102ab0 0x7f0b7c102e90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0b78009c20 tx=0x7f0b7802aa00 comp rx=0 tx=0).stop 2024-09-19T06:16:38.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.326+0000 7f0b83f84640 1 -- 172.21.15.53:0/567823801 shutdown_connections 2024-09-19T06:16:38.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.326+0000 7f0b83f84640 1 --2- 172.21.15.53:0/567823801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b7c1033d0 0x7f0b7c10b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.326+0000 7f0b83f84640 1 --2- 172.21.15.53:0/567823801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b7c102ab0 0x7f0b7c102e90 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.326+0000 7f0b83f84640 1 -- 172.21.15.53:0/567823801 >> 172.21.15.53:0/567823801 conn(0x7f0b7c0fc550 msgr2=0x7f0b7c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:38.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.326+0000 7f0b83f84640 1 -- 172.21.15.53:0/567823801 shutdown_connections 2024-09-19T06:16:38.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.326+0000 7f0b83f84640 1 -- 172.21.15.53:0/567823801 wait complete. 2024-09-19T06:16:38.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.326+0000 7f0b83f84640 1 Processor -- start 2024-09-19T06:16:38.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.326+0000 7f0b83f84640 1 -- start start 2024-09-19T06:16:38.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.327+0000 7f0b83f84640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b7c102ab0 0x7f0b7c19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:38.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.327+0000 7f0b83f84640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b7c1033d0 0x7f0b7c19dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:38.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.327+0000 7f0b83f84640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c10ed50 con 0x7f0b7c102ab0 2024-09-19T06:16:38.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.327+0000 7f0b83f84640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c10ebd0 con 0x7f0b7c1033d0 2024-09-19T06:16:38.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.327+0000 7f0b81d20640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b7c102ab0 0x7f0b7c19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:38.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.327+0000 7f0b81d20640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b7c102ab0 0x7f0b7c19d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59952/0 (socket says 172.21.15.53:59952) 2024-09-19T06:16:38.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.327+0000 7f0b81d20640 1 -- 172.21.15.53:0/3325507732 learned_addr learned my addr 172.21.15.53:0/3325507732 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:38.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.327+0000 7f0b8151f640 1 --2- 172.21.15.53:0/3325507732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b7c1033d0 0x7f0b7c19dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:38.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.327+0000 7f0b81d20640 1 -- 172.21.15.53:0/3325507732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b7c1033d0 msgr2=0x7f0b7c19dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:38.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.327+0000 7f0b81d20640 1 --2- 172.21.15.53:0/3325507732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b7c1033d0 0x7f0b7c19dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.327+0000 7f0b81d20640 1 -- 172.21.15.53:0/3325507732 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7c197770 con 0x7f0b7c102ab0 2024-09-19T06:16:38.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.327+0000 7f0b81d20640 1 --2- 172.21.15.53:0/3325507732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b7c102ab0 0x7f0b7c19d6a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0b78005cc0 tx=0x7f0b7802e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:38.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.327+0000 7f0b72ffd640 1 -- 172.21.15.53:0/3325507732 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0b78047070 con 0x7f0b7c102ab0 2024-09-19T06:16:38.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.327+0000 7f0b83f84640 1 -- 172.21.15.53:0/3325507732 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b7c197a00 con 0x7f0b7c102ab0 2024-09-19T06:16:38.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.327+0000 7f0b72ffd640 1 -- 172.21.15.53:0/3325507732 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b78003e20 con 0x7f0b7c102ab0 2024-09-19T06:16:38.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.327+0000 7f0b72ffd640 1 -- 172.21.15.53:0/3325507732 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0b78033050 con 0x7f0b7c102ab0 2024-09-19T06:16:38.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.327+0000 7f0b83f84640 1 -- 172.21.15.53:0/3325507732 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b7c197f30 con 0x7f0b7c102ab0 2024-09-19T06:16:38.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.328+0000 7f0b83f84640 1 -- 172.21.15.53:0/3325507732 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b7c06a580 con 0x7f0b7c102ab0 2024-09-19T06:16:38.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.329+0000 7f0b72ffd640 1 -- 172.21.15.53:0/3325507732 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f0b7804a070 con 0x7f0b7c102ab0 2024-09-19T06:16:38.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.329+0000 7f0b72ffd640 1 --2- 172.21.15.53:0/3325507732 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f0b6806e2b0 0x7f0b68070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:38.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.329+0000 7f0b72ffd640 1 -- 172.21.15.53:0/3325507732 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7138+0+0 (secure 0 0 0) 0x7f0b7803e030 con 0x7f0b7c102ab0 2024-09-19T06:16:38.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.329+0000 7f0b8151f640 1 --2- 172.21.15.53:0/3325507732 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f0b6806e2b0 0x7f0b68070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:38.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.330+0000 7f0b8151f640 1 --2- 172.21.15.53:0/3325507732 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f0b6806e2b0 0x7f0b68070770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c198fc0 tx=0x7f0b6c00b2e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:38.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.334+0000 7f0b72ffd640 1 -- 172.21.15.53:0/3325507732 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0b78080940 con 0x7f0b7c102ab0 2024-09-19T06:16:38.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.554+0000 7f0b83f84640 1 -- 172.21.15.53:0/3325507732 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b7c198ca0 con 0x7f0b7c102ab0 2024-09-19T06:16:38.556 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:38 smithi053 ceph-mon[27968]: Saving service mds.bar spec with placement count:2 2024-09-19T06:16:38.556 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:38 smithi053 ceph-mon[27968]: osdmap e68: 8 total, 8 up, 8 in 2024-09-19T06:16:38.556 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:38 smithi053 ceph-mon[27968]: mds.? [v2:172.21.15.53:6834/2573849324,v1:172.21.15.53:6835/2573849324] up:active 2024-09-19T06:16:38.557 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:38 smithi053 ceph-mon[27968]: fsmap foo:1 bar:0 {foo:0=foo.smithi053.rdifzk=up:active} 2024-09-19T06:16:38.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.555+0000 7f0b72ffd640 1 -- 172.21.15.53:0/3325507732 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+4059 (secure 0 0 0) 0x7f0b78078030 con 0x7f0b7c102ab0 2024-09-19T06:16:38.557 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:16:38.557 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":4,"btime":"2024-09-19T06:16:37:805074+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":4,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T06:16:32.878153+0000","modified":"2024-09-19T06:16:37.805067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":14510},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14510":{"gid":14510,"name":"foo.smithi053.rdifzk","rank":0,"incarnation":3,"state":"up:active","state_seq":2,"addr":"172.21.15.53:6835/2573849324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2573849324},{"type":"v1","addr":"172.21.15.53:6835","nonce":2573849324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"foo","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":14510,"qdb_cluster":[14510]},"id":1},{"mdsmap":{"epoch":3,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T06:16:36.000321+0000","modified":"2024-09-19T06:16:36.000321+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[],"info":{},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"bar","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-09-19T06:16:38.558 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 4 2024-09-19T06:16:38.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.557+0000 7f0b83f84640 1 -- 172.21.15.53:0/3325507732 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f0b6806e2b0 msgr2=0x7f0b68070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:38.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.557+0000 7f0b83f84640 1 --2- 172.21.15.53:0/3325507732 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f0b6806e2b0 0x7f0b68070770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c198fc0 tx=0x7f0b6c00b2e0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.557+0000 7f0b83f84640 1 -- 172.21.15.53:0/3325507732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b7c102ab0 msgr2=0x7f0b7c19d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:38.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.557+0000 7f0b83f84640 1 --2- 172.21.15.53:0/3325507732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b7c102ab0 0x7f0b7c19d6a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0b78005cc0 tx=0x7f0b7802e650 comp rx=0 tx=0).stop 2024-09-19T06:16:38.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.557+0000 7f0b83f84640 1 -- 172.21.15.53:0/3325507732 shutdown_connections 2024-09-19T06:16:38.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.557+0000 7f0b83f84640 1 --2- 172.21.15.53:0/3325507732 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f0b6806e2b0 0x7f0b68070770 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.557+0000 7f0b83f84640 1 --2- 172.21.15.53:0/3325507732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b7c1033d0 0x7f0b7c19dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.557+0000 7f0b83f84640 1 --2- 172.21.15.53:0/3325507732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b7c102ab0 0x7f0b7c19d6a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.557+0000 7f0b83f84640 1 -- 172.21.15.53:0/3325507732 >> 172.21.15.53:0/3325507732 conn(0x7f0b7c0fc550 msgr2=0x7f0b7c107dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:38.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.557+0000 7f0b83f84640 1 -- 172.21.15.53:0/3325507732 shutdown_connections 2024-09-19T06:16:38.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:38.557+0000 7f0b83f84640 1 -- 172.21.15.53:0/3325507732 wait complete. 2024-09-19T06:16:38.721 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:38 smithi155 ceph-mon[35292]: Saving service mds.bar spec with placement count:2 2024-09-19T06:16:38.721 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:38 smithi155 ceph-mon[35292]: osdmap e68: 8 total, 8 up, 8 in 2024-09-19T06:16:38.721 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:38 smithi155 ceph-mon[35292]: mds.? [v2:172.21.15.53:6834/2573849324,v1:172.21.15.53:6835/2573849324] up:active 2024-09-19T06:16:38.721 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:38 smithi155 ceph-mon[35292]: fsmap foo:1 bar:0 {foo:0=foo.smithi053.rdifzk=up:active} 2024-09-19T06:16:39.144 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:16:39.431 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:39.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:39 smithi155 ceph-mon[35292]: osdmap e69: 8 total, 8 up, 8 in 2024-09-19T06:16:39.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:39 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/3325507732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:16:39.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:39 smithi155 ceph-mon[35292]: pgmap v212: 129 pgs: 112 active+clean, 11 creating+peering, 6 unknown; 578 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-19T06:16:39.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:39 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:39.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:39 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:39.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:39 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:39.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:39 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:39.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:39 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:39.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:39 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:16:39.795 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:39 smithi053 ceph-mon[27968]: osdmap e69: 8 total, 8 up, 8 in 2024-09-19T06:16:39.795 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:39 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/3325507732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:16:39.795 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:39 smithi053 ceph-mon[27968]: pgmap v212: 129 pgs: 112 active+clean, 11 creating+peering, 6 unknown; 578 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-19T06:16:39.795 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:39 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:39.795 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:39 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:39.795 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:39 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:39.795 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:39 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:39.796 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:39 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:39.796 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:39 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:16:39.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.874+0000 7f9619242640 1 -- 172.21.15.53:0/556005024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9614100800 msgr2=0x7f9614100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:39.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.874+0000 7f9619242640 1 --2- 172.21.15.53:0/556005024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9614100800 0x7f9614100c60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f95fc009880 tx=0x7f95fc02a4a0 comp rx=0 tx=0).stop 2024-09-19T06:16:39.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.875+0000 7f9619242640 1 -- 172.21.15.53:0/556005024 shutdown_connections 2024-09-19T06:16:39.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.875+0000 7f9619242640 1 --2- 172.21.15.53:0/556005024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9614100800 0x7f9614100c60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:39.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.875+0000 7f9619242640 1 --2- 172.21.15.53:0/556005024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9614106800 0x7f9614106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:39.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.875+0000 7f9619242640 1 -- 172.21.15.53:0/556005024 >> 172.21.15.53:0/556005024 conn(0x7f96140fc520 msgr2=0x7f96140fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:39.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.875+0000 7f9619242640 1 -- 172.21.15.53:0/556005024 shutdown_connections 2024-09-19T06:16:39.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.875+0000 7f9619242640 1 -- 172.21.15.53:0/556005024 wait complete. 2024-09-19T06:16:39.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.875+0000 7f9619242640 1 Processor -- start 2024-09-19T06:16:39.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.876+0000 7f9619242640 1 -- start start 2024-09-19T06:16:39.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.876+0000 7f9619242640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9614100800 0x7f9614198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:39.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.876+0000 7f9619242640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9614106800 0x7f9614198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:39.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.876+0000 7f9619242640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f961410dc20 con 0x7f9614100800 2024-09-19T06:16:39.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.876+0000 7f9619242640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f961410daa0 con 0x7f9614106800 2024-09-19T06:16:39.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.876+0000 7f9612d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9614100800 0x7f9614198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:39.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.876+0000 7f9612d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9614100800 0x7f9614198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59956/0 (socket says 172.21.15.53:59956) 2024-09-19T06:16:39.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.876+0000 7f9612d9d640 1 -- 172.21.15.53:0/1172465647 learned_addr learned my addr 172.21.15.53:0/1172465647 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:39.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.876+0000 7f960bfff640 1 --2- 172.21.15.53:0/1172465647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9614106800 0x7f9614198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:39.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.876+0000 7f9612d9d640 1 -- 172.21.15.53:0/1172465647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9614106800 msgr2=0x7f9614198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:39.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.876+0000 7f9612d9d640 1 --2- 172.21.15.53:0/1172465647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9614106800 0x7f9614198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:39.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.876+0000 7f9612d9d640 1 -- 172.21.15.53:0/1172465647 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9614199240 con 0x7f9614100800 2024-09-19T06:16:39.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.876+0000 7f9612d9d640 1 --2- 172.21.15.53:0/1172465647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9614100800 0x7f9614198740 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f960400d8d0 tx=0x7f960400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:39.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.876+0000 7f9610d99640 1 -- 172.21.15.53:0/1172465647 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9604017030 con 0x7f9614100800 2024-09-19T06:16:39.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.877+0000 7f9619242640 1 -- 172.21.15.53:0/1172465647 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9614199530 con 0x7f9614100800 2024-09-19T06:16:39.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.877+0000 7f9610d99640 1 -- 172.21.15.53:0/1172465647 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9604004020 con 0x7f9614100800 2024-09-19T06:16:39.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.877+0000 7f9610d99640 1 -- 172.21.15.53:0/1172465647 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9604005580 con 0x7f9614100800 2024-09-19T06:16:39.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.877+0000 7f9619242640 1 -- 172.21.15.53:0/1172465647 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96141a1af0 con 0x7f9614100800 2024-09-19T06:16:39.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.878+0000 7f9619242640 1 -- 172.21.15.53:0/1172465647 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9614101f40 con 0x7f9614100800 2024-09-19T06:16:39.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.878+0000 7f9610d99640 1 -- 172.21.15.53:0/1172465647 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f96040041d0 con 0x7f9614100800 2024-09-19T06:16:39.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.878+0000 7f9610d99640 1 --2- 172.21.15.53:0/1172465647 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f960006e2b0 0x7f9600070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:39.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.879+0000 7f9610d99640 1 -- 172.21.15.53:0/1172465647 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7138+0+0 (secure 0 0 0) 0x7f9604018070 con 0x7f9614100800 2024-09-19T06:16:39.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.879+0000 7f960bfff640 1 --2- 172.21.15.53:0/1172465647 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f960006e2b0 0x7f9600070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:39.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.879+0000 7f960bfff640 1 --2- 172.21.15.53:0/1172465647 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f960006e2b0 0x7f9600070770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f95fc000c00 tx=0x7f95fc0058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:39.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:39.883+0000 7f9610d99640 1 -- 172.21.15.53:0/1172465647 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f960405b760 con 0x7f9614100800 2024-09-19T06:16:40.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:40.102+0000 7f9619242640 1 -- 172.21.15.53:0/1172465647 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9614100c60 con 0x7f9614100800 2024-09-19T06:16:40.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:40.103+0000 7f9610d99640 1 -- 172.21.15.53:0/1172465647 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4918 (secure 0 0 0) 0x7f960405f410 con 0x7f9614100800 2024-09-19T06:16:40.105 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:16:40.105 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":6,"btime":"2024-09-19T06:16:39:469727+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":4,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T06:16:32.878153+0000","modified":"2024-09-19T06:16:37.805067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":14510},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14510":{"gid":14510,"name":"foo.smithi053.rdifzk","rank":0,"incarnation":3,"state":"up:active","state_seq":2,"addr":"172.21.15.53:6835/2573849324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2573849324},{"type":"v1","addr":"172.21.15.53:6835","nonce":2573849324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"foo","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14510,"qdb_cluster":[14510]},"id":1},{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T06:16:36.000321+0000","modified":"2024-09-19T06:16:39.469710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24319":{"gid":24319,"name":"foo.smithi155.dsbavw","rank":0,"incarnation":6,"state":"up:creating","state_seq":1,"addr":"172.21.15.155:6833/390026217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":390026217},{"type":"v1","addr":"172.21.15.155:6833","nonce":390026217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"bar","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-09-19T06:16:40.105 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 6 2024-09-19T06:16:40.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:40.105+0000 7f9619242640 1 -- 172.21.15.53:0/1172465647 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f960006e2b0 msgr2=0x7f9600070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:40.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:40.105+0000 7f9619242640 1 --2- 172.21.15.53:0/1172465647 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f960006e2b0 0x7f9600070770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f95fc000c00 tx=0x7f95fc0058d0 comp rx=0 tx=0).stop 2024-09-19T06:16:40.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:40.105+0000 7f9619242640 1 -- 172.21.15.53:0/1172465647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9614100800 msgr2=0x7f9614198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:40.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:40.105+0000 7f9619242640 1 --2- 172.21.15.53:0/1172465647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9614100800 0x7f9614198740 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f960400d8d0 tx=0x7f960400dd90 comp rx=0 tx=0).stop 2024-09-19T06:16:40.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:40.105+0000 7f9619242640 1 -- 172.21.15.53:0/1172465647 shutdown_connections 2024-09-19T06:16:40.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:40.105+0000 7f9619242640 1 --2- 172.21.15.53:0/1172465647 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f960006e2b0 0x7f9600070770 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:40.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:40.105+0000 7f9619242640 1 --2- 172.21.15.53:0/1172465647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9614106800 0x7f9614198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:40.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:40.105+0000 7f9619242640 1 --2- 172.21.15.53:0/1172465647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9614100800 0x7f9614198740 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:40.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:40.105+0000 7f9619242640 1 -- 172.21.15.53:0/1172465647 >> 172.21.15.53:0/1172465647 conn(0x7f96140fc520 msgr2=0x7f96140fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:40.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:40.105+0000 7f9619242640 1 -- 172.21.15.53:0/1172465647 shutdown_connections 2024-09-19T06:16:40.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:40.105+0000 7f9619242640 1 -- 172.21.15.53:0/1172465647 wait complete. 2024-09-19T06:16:40.623 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:40 smithi053 ceph-mon[27968]: mds.? [v2:172.21.15.155:6832/390026217,v1:172.21.15.155:6833/390026217] up:boot 2024-09-19T06:16:40.623 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:40 smithi053 ceph-mon[27968]: daemon mds.foo.smithi155.dsbavw assigned to filesystem bar as rank 0 (now has 1 ranks) 2024-09-19T06:16:40.623 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:40 smithi053 ceph-mon[27968]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-09-19T06:16:40.623 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:40 smithi053 ceph-mon[27968]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T06:16:40.624 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:40 smithi053 ceph-mon[27968]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T06:16:40.624 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:40 smithi053 ceph-mon[27968]: fsmap foo:1 bar:0 {foo:0=foo.smithi053.rdifzk=up:active} 1 up:standby 2024-09-19T06:16:40.624 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:40 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mds metadata", "who": "foo.smithi155.dsbavw"}]: dispatch 2024-09-19T06:16:40.624 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:40 smithi053 ceph-mon[27968]: fsmap foo:1 bar:1 {bar:0=foo.smithi155.dsbavw=up:creating,foo:0=foo.smithi053.rdifzk=up:active} 2024-09-19T06:16:40.624 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:40 smithi053 ceph-mon[27968]: daemon mds.foo.smithi155.dsbavw is now active in filesystem bar as rank 0 2024-09-19T06:16:40.624 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:40 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/1172465647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:16:40.702 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:40 smithi155 ceph-mon[35292]: mds.? [v2:172.21.15.155:6832/390026217,v1:172.21.15.155:6833/390026217] up:boot 2024-09-19T06:16:40.703 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:40 smithi155 ceph-mon[35292]: daemon mds.foo.smithi155.dsbavw assigned to filesystem bar as rank 0 (now has 1 ranks) 2024-09-19T06:16:40.703 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:40 smithi155 ceph-mon[35292]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-09-19T06:16:40.703 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:40 smithi155 ceph-mon[35292]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T06:16:40.703 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:40 smithi155 ceph-mon[35292]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T06:16:40.703 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:40 smithi155 ceph-mon[35292]: fsmap foo:1 bar:0 {foo:0=foo.smithi053.rdifzk=up:active} 1 up:standby 2024-09-19T06:16:40.703 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:40 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mds metadata", "who": "foo.smithi155.dsbavw"}]: dispatch 2024-09-19T06:16:40.703 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:40 smithi155 ceph-mon[35292]: fsmap foo:1 bar:1 {bar:0=foo.smithi155.dsbavw=up:creating,foo:0=foo.smithi053.rdifzk=up:active} 2024-09-19T06:16:40.703 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:40 smithi155 ceph-mon[35292]: daemon mds.foo.smithi155.dsbavw is now active in filesystem bar as rank 0 2024-09-19T06:16:40.703 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:40 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/1172465647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:16:41.268 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:16:41.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:41 smithi053 ceph-mon[27968]: mds.? [v2:172.21.15.155:6832/390026217,v1:172.21.15.155:6833/390026217] up:active 2024-09-19T06:16:41.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:41 smithi053 ceph-mon[27968]: fsmap foo:1 bar:1 {bar:0=foo.smithi155.dsbavw=up:active,foo:0=foo.smithi053.rdifzk=up:active} 2024-09-19T06:16:41.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:41 smithi053 ceph-mon[27968]: pgmap v213: 129 pgs: 118 active+clean, 11 creating+peering; 578 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s wr, 7 op/s 2024-09-19T06:16:41.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:41 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:41.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:41 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:41.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:41 smithi155 ceph-mon[35292]: mds.? [v2:172.21.15.155:6832/390026217,v1:172.21.15.155:6833/390026217] up:active 2024-09-19T06:16:41.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:41 smithi155 ceph-mon[35292]: fsmap foo:1 bar:1 {bar:0=foo.smithi155.dsbavw=up:active,foo:0=foo.smithi053.rdifzk=up:active} 2024-09-19T06:16:41.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:41 smithi155 ceph-mon[35292]: pgmap v213: 129 pgs: 118 active+clean, 11 creating+peering; 578 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s wr, 7 op/s 2024-09-19T06:16:41.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:41 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:41.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:41 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:42.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:42 smithi053 ceph-mon[27968]: mds.? [v2:172.21.15.53:6834/2573849324,v1:172.21.15.53:6835/2573849324] up:active 2024-09-19T06:16:42.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:42 smithi053 ceph-mon[27968]: fsmap foo:1 bar:1 {bar:0=foo.smithi155.dsbavw=up:active,foo:0=foo.smithi053.rdifzk=up:active} 2024-09-19T06:16:42.709 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:42.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:42 smithi155 ceph-mon[35292]: mds.? [v2:172.21.15.53:6834/2573849324,v1:172.21.15.53:6835/2573849324] up:active 2024-09-19T06:16:42.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:42 smithi155 ceph-mon[35292]: fsmap foo:1 bar:1 {bar:0=foo.smithi155.dsbavw=up:active,foo:0=foo.smithi053.rdifzk=up:active} 2024-09-19T06:16:43.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.101+0000 7f653a31e640 1 -- 172.21.15.53:0/3436673020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6534102ac0 msgr2=0x7f6534102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:43.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.101+0000 7f653a31e640 1 --2- 172.21.15.53:0/3436673020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6534102ac0 0x7f6534102ea0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6524009880 tx=0x7f652402a530 comp rx=0 tx=0).stop 2024-09-19T06:16:43.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.101+0000 7f653a31e640 1 -- 172.21.15.53:0/3436673020 shutdown_connections 2024-09-19T06:16:43.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.101+0000 7f653a31e640 1 --2- 172.21.15.53:0/3436673020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65341033e0 0x7f653410b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.101+0000 7f653a31e640 1 --2- 172.21.15.53:0/3436673020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6534102ac0 0x7f6534102ea0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.101+0000 7f653a31e640 1 -- 172.21.15.53:0/3436673020 >> 172.21.15.53:0/3436673020 conn(0x7f65340fc520 msgr2=0x7f65340fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:43.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.101+0000 7f653a31e640 1 -- 172.21.15.53:0/3436673020 shutdown_connections 2024-09-19T06:16:43.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.101+0000 7f653a31e640 1 -- 172.21.15.53:0/3436673020 wait complete. 2024-09-19T06:16:43.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.101+0000 7f653a31e640 1 Processor -- start 2024-09-19T06:16:43.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.102+0000 7f653a31e640 1 -- start start 2024-09-19T06:16:43.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.102+0000 7f653a31e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6534102ac0 0x7f653419d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:43.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.102+0000 7f653a31e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65341033e0 0x7f653419dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:43.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.102+0000 7f653a31e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f653410edc0 con 0x7f6534102ac0 2024-09-19T06:16:43.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.102+0000 7f653a31e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f653410ec40 con 0x7f65341033e0 2024-09-19T06:16:43.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.102+0000 7f6533fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6534102ac0 0x7f653419d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:43.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.102+0000 7f6533fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6534102ac0 0x7f653419d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45988/0 (socket says 172.21.15.53:45988) 2024-09-19T06:16:43.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.102+0000 7f6533fff640 1 -- 172.21.15.53:0/1001570738 learned_addr learned my addr 172.21.15.53:0/1001570738 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:43.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.102+0000 7f652bfff640 1 --2- 172.21.15.53:0/1001570738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65341033e0 0x7f653419dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:43.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.102+0000 7f6533fff640 1 -- 172.21.15.53:0/1001570738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65341033e0 msgr2=0x7f653419dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:43.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.102+0000 7f6533fff640 1 --2- 172.21.15.53:0/1001570738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65341033e0 0x7f653419dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.102+0000 7f6533fff640 1 -- 172.21.15.53:0/1001570738 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f653419e350 con 0x7f6534102ac0 2024-09-19T06:16:43.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.102+0000 7f652bfff640 1 --2- 172.21.15.53:0/1001570738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65341033e0 0x7f653419dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:16:43.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.102+0000 7f6533fff640 1 --2- 172.21.15.53:0/1001570738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6534102ac0 0x7f653419d760 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6524006010 tx=0x7f65240048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:43.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.103+0000 7f6531ffb640 1 -- 172.21.15.53:0/1001570738 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f652403f070 con 0x7f6534102ac0 2024-09-19T06:16:43.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.103+0000 7f653a31e640 1 -- 172.21.15.53:0/1001570738 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65341979e0 con 0x7f6534102ac0 2024-09-19T06:16:43.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.103+0000 7f6531ffb640 1 -- 172.21.15.53:0/1001570738 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6524003aa0 con 0x7f6534102ac0 2024-09-19T06:16:43.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.103+0000 7f6531ffb640 1 -- 172.21.15.53:0/1001570738 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6524035030 con 0x7f6534102ac0 2024-09-19T06:16:43.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.103+0000 7f653a31e640 1 -- 172.21.15.53:0/1001570738 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6534197ee0 con 0x7f6534102ac0 2024-09-19T06:16:43.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.104+0000 7f653a31e640 1 -- 172.21.15.53:0/1001570738 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f653406aa90 con 0x7f6534102ac0 2024-09-19T06:16:43.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.104+0000 7f6531ffb640 1 -- 172.21.15.53:0/1001570738 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f6524042070 con 0x7f6534102ac0 2024-09-19T06:16:43.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.105+0000 7f6531ffb640 1 --2- 172.21.15.53:0/1001570738 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f652006e2b0 0x7f6520070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:43.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.105+0000 7f6531ffb640 1 -- 172.21.15.53:0/1001570738 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7138+0+0 (secure 0 0 0) 0x7f65240af400 con 0x7f6534102ac0 2024-09-19T06:16:43.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.105+0000 7f652bfff640 1 --2- 172.21.15.53:0/1001570738 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f652006e2b0 0x7f6520070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:43.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.105+0000 7f652bfff640 1 --2- 172.21.15.53:0/1001570738 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f652006e2b0 0x7f6520070770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f651c006fd0 tx=0x7f651c008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:43.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.109+0000 7f6531ffb640 1 -- 172.21.15.53:0/1001570738 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f652407c640 con 0x7f6534102ac0 2024-09-19T06:16:43.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.285+0000 7f653a31e640 1 -- 172.21.15.53:0/1001570738 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6534198190 con 0x7f6534102ac0 2024-09-19T06:16:43.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.288+0000 7f6531ffb640 1 -- 172.21.15.53:0/1001570738 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+21340 (secure 0 0 0) 0x7f65240802f0 con 0x7f6534102ac0 2024-09-19T06:16:43.290 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:16:43.290 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":69,"fsid":"9f2876d0-764d-11ef-bceb-c7b262605968","created":"2024-09-19T06:09:26.797853+0000","modified":"2024-09-19T06:16:38.457146+0000","last_up_change":"2024-09-19T06:15:47.809832+0000","last_in_change":"2024-09-19T06:15:26.420480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T06:13:39.241335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.foo.meta","create_time":"2024-09-19T06:16:31.363649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"foo"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.foo.data","create_time":"2024-09-19T06:16:32.311547+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"foo"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs.bar.meta","create_time":"2024-09-19T06:16:34.984973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"bar"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs.bar.data","create_time":"2024-09-19T06:16:35.435954+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"bar"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"35349d4a-0bde-46f6-8d5f-a019f0f89095","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1456825097},{"type":"v1","addr":"172.21.15.53:6803","nonce":1456825097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1456825097},{"type":"v1","addr":"172.21.15.53:6805","nonce":1456825097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1456825097},{"type":"v1","addr":"172.21.15.53:6809","nonce":1456825097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1456825097},{"type":"v1","addr":"172.21.15.53:6807","nonce":1456825097}]},"public_addr":"172.21.15.53:6803/1456825097","cluster_addr":"172.21.15.53:6805/1456825097","heartbeat_back_addr":"172.21.15.53:6809/1456825097","heartbeat_front_addr":"172.21.15.53:6807/1456825097","state":["exists","up"]},{"osd":1,"uuid":"832c2772-a460-4691-8db7-ad1d0c48cc9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3638097004},{"type":"v1","addr":"172.21.15.53:6811","nonce":3638097004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3638097004},{"type":"v1","addr":"172.21.15.53:6813","nonce":3638097004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3638097004},{"type":"v1","addr":"172.21.15.53:6817","nonce":3638097004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3638097004},{"type":"v1","addr":"172.21.15.53:6815","nonce":3638097004}]},"public_addr":"172.21.15.53:6811/3638097004","cluster_addr":"172.21.15.53:6813/3638097004","heartbeat_back_addr":"172.21.15.53:6817/3638097004","heartbeat_front_addr":"172.21.15.53:6815/3638097004","state":["exists","up"]},{"osd":2,"uuid":"57282117-559f-4495-b85f-90b68b2439ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":1988602192},{"type":"v1","addr":"172.21.15.53:6819","nonce":1988602192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":1988602192},{"type":"v1","addr":"172.21.15.53:6821","nonce":1988602192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1988602192},{"type":"v1","addr":"172.21.15.53:6825","nonce":1988602192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":1988602192},{"type":"v1","addr":"172.21.15.53:6823","nonce":1988602192}]},"public_addr":"172.21.15.53:6819/1988602192","cluster_addr":"172.21.15.53:6821/1988602192","heartbeat_back_addr":"172.21.15.53:6825/1988602192","heartbeat_front_addr":"172.21.15.53:6823/1988602192","state":["exists","up"]},{"osd":3,"uuid":"e6bf8cf0-2419-4c2a-b562-9221a117050c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2604012854},{"type":"v1","addr":"172.21.15.53:6827","nonce":2604012854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2604012854},{"type":"v1","addr":"172.21.15.53:6829","nonce":2604012854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2604012854},{"type":"v1","addr":"172.21.15.53:6833","nonce":2604012854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2604012854},{"type":"v1","addr":"172.21.15.53:6831","nonce":2604012854}]},"public_addr":"172.21.15.53:6827/2604012854","cluster_addr":"172.21.15.53:6829/2604012854","heartbeat_back_addr":"172.21.15.53:6833/2604012854","heartbeat_front_addr":"172.21.15.53:6831/2604012854","state":["exists","up"]},{"osd":4,"uuid":"7adedf0a-6fbc-42bc-8996-ba482b638929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2046029108},{"type":"v1","addr":"172.21.15.155:6801","nonce":2046029108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2046029108},{"type":"v1","addr":"172.21.15.155:6803","nonce":2046029108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2046029108},{"type":"v1","addr":"172.21.15.155:6807","nonce":2046029108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2046029108},{"type":"v1","addr":"172.21.15.155:6805","nonce":2046029108}]},"public_addr":"172.21.15.155:6801/2046029108","cluster_addr":"172.21.15.155:6803/2046029108","heartbeat_back_addr":"172.21.15.155:6807/2046029108","heartbeat_front_addr":"172.21.15.155:6805/2046029108","state":["exists","up"]},{"osd":5,"uuid":"c664e57c-20b5-46d4-8830-70780a91d6dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":464507356},{"type":"v1","addr":"172.21.15.155:6809","nonce":464507356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":464507356},{"type":"v1","addr":"172.21.15.155:6811","nonce":464507356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":464507356},{"type":"v1","addr":"172.21.15.155:6815","nonce":464507356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":464507356},{"type":"v1","addr":"172.21.15.155:6813","nonce":464507356}]},"public_addr":"172.21.15.155:6809/464507356","cluster_addr":"172.21.15.155:6811/464507356","heartbeat_back_addr":"172.21.15.155:6815/464507356","heartbeat_front_addr":"172.21.15.155:6813/464507356","state":["exists","up"]},{"osd":6,"uuid":"bc8a88f5-75ab-4482-88e0-5cc5e6f06db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2574655677},{"type":"v1","addr":"172.21.15.155:6817","nonce":2574655677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2574655677},{"type":"v1","addr":"172.21.15.155:6819","nonce":2574655677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2574655677},{"type":"v1","addr":"172.21.15.155:6823","nonce":2574655677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2574655677},{"type":"v1","addr":"172.21.15.155:6821","nonce":2574655677}]},"public_addr":"172.21.15.155:6817/2574655677","cluster_addr":"172.21.15.155:6819/2574655677","heartbeat_back_addr":"172.21.15.155:6823/2574655677","heartbeat_front_addr":"172.21.15.155:6821/2574655677","state":["exists","up"]},{"osd":7,"uuid":"b91759fe-f27c-4ab2-9bd3-b5961b358d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2269161534},{"type":"v1","addr":"172.21.15.155:6825","nonce":2269161534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2269161534},{"type":"v1","addr":"172.21.15.155:6827","nonce":2269161534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2269161534},{"type":"v1","addr":"172.21.15.155:6831","nonce":2269161534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2269161534},{"type":"v1","addr":"172.21.15.155:6829","nonce":2269161534}]},"public_addr":"172.21.15.155:6825/2269161534","cluster_addr":"172.21.15.155:6827/2269161534","heartbeat_back_addr":"172.21.15.155:6831/2269161534","heartbeat_front_addr":"172.21.15.155:6829/2269161534","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:12:43.760347+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:13:10.493898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:13:37.244856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:14:05.006717+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:14:29.801028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:14:55.095341+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:15:20.683202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:15:45.635704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6801/3684090301":"2024-09-20T06:11:28.721009+0000","172.21.15.53:6800/475342876":"2024-09-20T06:09:58.271775+0000","172.21.15.53:6801/475342876":"2024-09-20T06:09:58.271775+0000","172.21.15.53:0/3753705881":"2024-09-20T06:11:28.721009+0000","172.21.15.53:0/4216592427":"2024-09-20T06:09:58.271775+0000","172.21.15.53:0/1618433917":"2024-09-20T06:10:30.209033+0000","172.21.15.53:0/623598636":"2024-09-20T06:09:58.271775+0000","172.21.15.53:6800/3684090301":"2024-09-20T06:11:28.721009+0000","172.21.15.53:6800/681468213":"2024-09-20T06:10:30.209033+0000","172.21.15.53:0/597749918":"2024-09-20T06:10:30.209033+0000","172.21.15.53:0/21853845":"2024-09-20T06:11:28.721009+0000","172.21.15.53:0/3515303955":"2024-09-20T06:09:58.271775+0000","172.21.15.53:6801/681468213":"2024-09-20T06:10:30.209033+0000","172.21.15.53:0/1729980812":"2024-09-20T06:11:28.721009+0000","172.21.15.53:0/372276981":"2024-09-20T06:10:30.209033+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:16:43.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.290+0000 7f653a31e640 1 -- 172.21.15.53:0/1001570738 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f652006e2b0 msgr2=0x7f6520070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:43.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.290+0000 7f653a31e640 1 --2- 172.21.15.53:0/1001570738 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f652006e2b0 0x7f6520070770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f651c006fd0 tx=0x7f651c008040 comp rx=0 tx=0).stop 2024-09-19T06:16:43.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.290+0000 7f653a31e640 1 -- 172.21.15.53:0/1001570738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6534102ac0 msgr2=0x7f653419d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:43.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.290+0000 7f653a31e640 1 --2- 172.21.15.53:0/1001570738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6534102ac0 0x7f653419d760 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6524006010 tx=0x7f65240048f0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.290+0000 7f653a31e640 1 -- 172.21.15.53:0/1001570738 shutdown_connections 2024-09-19T06:16:43.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.290+0000 7f653a31e640 1 --2- 172.21.15.53:0/1001570738 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f652006e2b0 0x7f6520070770 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.290+0000 7f653a31e640 1 --2- 172.21.15.53:0/1001570738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65341033e0 0x7f653419dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.290+0000 7f653a31e640 1 --2- 172.21.15.53:0/1001570738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6534102ac0 0x7f653419d760 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.290+0000 7f653a31e640 1 -- 172.21.15.53:0/1001570738 >> 172.21.15.53:0/1001570738 conn(0x7f65340fc520 msgr2=0x7f6534103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:43.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.290+0000 7f653a31e640 1 -- 172.21.15.53:0/1001570738 shutdown_connections 2024-09-19T06:16:43.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:43.290+0000 7f653a31e640 1 -- 172.21.15.53:0/1001570738 wait complete. 2024-09-19T06:16:43.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:43 smithi053 ceph-mon[27968]: pgmap v214: 129 pgs: 129 active+clean; 580 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 3.8 KiB/s wr, 9 op/s 2024-09-19T06:16:43.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:43 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/1001570738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:16:43.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:43 smithi155 ceph-mon[35292]: pgmap v214: 129 pgs: 129 active+clean; 580 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 3.8 KiB/s wr, 9 op/s 2024-09-19T06:16:43.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:43 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/1001570738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:16:44.080 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 4, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T06:16:32.878153+0000', 'modified': '2024-09-19T06:16:37.805067+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 14510}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14510': {'gid': 14510, 'name': 'foo.smithi053.rdifzk', 'rank': 0, 'incarnation': 3, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.53:6835/2573849324', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6834', 'nonce': 2573849324}, {'type': 'v1', 'addr': '172.21.15.53:6835', 'nonce': 2573849324}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'foo', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 14510, 'qdb_cluster': [14510]} 2024-09-19T06:16:44.080 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T06:16:44.081 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph tell mds.foo.smithi053.rdifzk status 2024-09-19T06:16:44.424 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:44.833 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:44 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:44.834 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:44 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:16:44.834 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:44 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:45.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:44 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:45.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:44 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:16:45.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:44 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:45.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.503+0000 7f62ea475640 1 -- 172.21.15.53:0/2033461859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62e4102ac0 msgr2=0x7f62e4102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:45.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.503+0000 7f62ea475640 1 --2- 172.21.15.53:0/2033461859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62e4102ac0 0x7f62e4102ea0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f62d4009880 tx=0x7f62d402a530 comp rx=0 tx=0).stop 2024-09-19T06:16:45.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.503+0000 7f62ea475640 1 -- 172.21.15.53:0/2033461859 shutdown_connections 2024-09-19T06:16:45.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.503+0000 7f62ea475640 1 --2- 172.21.15.53:0/2033461859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62e41033e0 0x7f62e410b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.503+0000 7f62ea475640 1 --2- 172.21.15.53:0/2033461859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62e4102ac0 0x7f62e4102ea0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.503+0000 7f62ea475640 1 -- 172.21.15.53:0/2033461859 >> 172.21.15.53:0/2033461859 conn(0x7f62e40fc520 msgr2=0x7f62e40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:45.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.503+0000 7f62ea475640 1 -- 172.21.15.53:0/2033461859 shutdown_connections 2024-09-19T06:16:45.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.503+0000 7f62ea475640 1 -- 172.21.15.53:0/2033461859 wait complete. 2024-09-19T06:16:45.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.504+0000 7f62ea475640 1 Processor -- start 2024-09-19T06:16:45.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.504+0000 7f62ea475640 1 -- start start 2024-09-19T06:16:45.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.504+0000 7f62ea475640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62e4102ac0 0x7f62e419d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:45.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.504+0000 7f62ea475640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62e41033e0 0x7f62e419db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:45.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.504+0000 7f62ea475640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f62e410ed60 con 0x7f62e4102ac0 2024-09-19T06:16:45.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.504+0000 7f62ea475640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f62e410ebe0 con 0x7f62e41033e0 2024-09-19T06:16:45.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.504+0000 7f62e3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62e4102ac0 0x7f62e419d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:45.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.504+0000 7f62e3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62e4102ac0 0x7f62e419d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46018/0 (socket says 172.21.15.53:46018) 2024-09-19T06:16:45.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.504+0000 7f62e3fff640 1 -- 172.21.15.53:0/1256177023 learned_addr learned my addr 172.21.15.53:0/1256177023 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:45.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.505+0000 7f62dbfff640 1 --2- 172.21.15.53:0/1256177023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62e41033e0 0x7f62e419db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:45.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.505+0000 7f62e3fff640 1 -- 172.21.15.53:0/1256177023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62e41033e0 msgr2=0x7f62e419db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:45.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.505+0000 7f62e3fff640 1 --2- 172.21.15.53:0/1256177023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62e41033e0 0x7f62e419db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.505+0000 7f62e3fff640 1 -- 172.21.15.53:0/1256177023 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e41976e0 con 0x7f62e4102ac0 2024-09-19T06:16:45.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.505+0000 7f62e3fff640 1 --2- 172.21.15.53:0/1256177023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62e4102ac0 0x7f62e419d610 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f62d4006010 tx=0x7f62d40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:45.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.505+0000 7f62e1ffb640 1 -- 172.21.15.53:0/1256177023 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f62d403f070 con 0x7f62e4102ac0 2024-09-19T06:16:45.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.505+0000 7f62e1ffb640 1 -- 172.21.15.53:0/1256177023 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62d4003aa0 con 0x7f62e4102ac0 2024-09-19T06:16:45.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.505+0000 7f62ea475640 1 -- 172.21.15.53:0/1256177023 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e4197970 con 0x7f62e4102ac0 2024-09-19T06:16:45.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.505+0000 7f62e1ffb640 1 -- 172.21.15.53:0/1256177023 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f62d4035030 con 0x7f62e4102ac0 2024-09-19T06:16:45.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.505+0000 7f62ea475640 1 -- 172.21.15.53:0/1256177023 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e4197ea0 con 0x7f62e4102ac0 2024-09-19T06:16:45.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.507+0000 7f62e1ffb640 1 -- 172.21.15.53:0/1256177023 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f62d4042070 con 0x7f62e4102ac0 2024-09-19T06:16:45.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.507+0000 7f62e1ffb640 1 --2- 172.21.15.53:0/1256177023 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f62d006e310 0x7f62d00707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:45.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.507+0000 7f62e1ffb640 1 -- 172.21.15.53:0/1256177023 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7138+0+0 (secure 0 0 0) 0x7f62d40af330 con 0x7f62e4102ac0 2024-09-19T06:16:45.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.507+0000 7f62dbfff640 1 --2- 172.21.15.53:0/1256177023 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f62d006e310 0x7f62d00707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:45.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.508+0000 7f62ec84e740 1 Processor -- start 2024-09-19T06:16:45.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.508+0000 7f62ec84e740 1 -- start start 2024-09-19T06:16:45.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.508+0000 7f62dbfff640 1 --2- 172.21.15.53:0/1256177023 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f62d006e310 0x7f62d00707d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f62e4198f30 tx=0x7f62cc005ec0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:45.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.508+0000 7f62ec84e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560924584430 0x560924584810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:45.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.508+0000 7f62ec84e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560924584d50 0x56092458f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:45.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.508+0000 7f62ec84e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5609244ea180 con 0x560924584d50 2024-09-19T06:16:45.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.508+0000 7f62ec84e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560924490d40 con 0x560924584430 2024-09-19T06:16:45.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.508+0000 7f62dbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560924584d50 0x56092458f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:45.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.508+0000 7f62dbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560924584d50 0x56092458f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46028/0 (socket says 172.21.15.53:46028) 2024-09-19T06:16:45.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.508+0000 7f62dbfff640 1 -- 172.21.15.53:0/2029275737 learned_addr learned my addr 172.21.15.53:0/2029275737 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:45.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.508+0000 7f62e3fff640 1 --2- 172.21.15.53:0/2029275737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560924584430 0x560924584810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:45.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.508+0000 7f62dbfff640 1 -- 172.21.15.53:0/2029275737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560924584430 msgr2=0x560924584810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:45.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.508+0000 7f62dbfff640 1 --2- 172.21.15.53:0/2029275737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560924584430 0x560924584810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.508+0000 7f62dbfff640 1 -- 172.21.15.53:0/2029275737 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56092425b420 con 0x560924584d50 2024-09-19T06:16:45.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.508+0000 7f62e3fff640 1 --2- 172.21.15.53:0/2029275737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560924584430 0x560924584810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:16:45.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.509+0000 7f62dbfff640 1 --2- 172.21.15.53:0/2029275737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560924584d50 0x56092458f0d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00afd0 tx=0x7f62cc011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e2733e4c710f39 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:45.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.509+0000 7f62da7fc640 1 -- 172.21.15.53:0/2029275737 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f62cc021070 con 0x560924584d50 2024-09-19T06:16:45.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.509+0000 7f62da7fc640 1 -- 172.21.15.53:0/2029275737 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62cc01d900 con 0x560924584d50 2024-09-19T06:16:45.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.509+0000 7f62da7fc640 1 -- 172.21.15.53:0/2029275737 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f62cc00f750 con 0x560924584d50 2024-09-19T06:16:45.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.509+0000 7f62ec84e740 1 -- 172.21.15.53:0/2029275737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560924584d50 msgr2=0x56092458f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:45.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.510+0000 7f62ec84e740 1 --2- 172.21.15.53:0/2029275737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560924584d50 0x56092458f0d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00afd0 tx=0x7f62cc011000 comp rx=0 tx=0).stop 2024-09-19T06:16:45.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.510+0000 7f62ec84e740 1 -- 172.21.15.53:0/2029275737 shutdown_connections 2024-09-19T06:16:45.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.510+0000 7f62ec84e740 1 --2- 172.21.15.53:0/2029275737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560924584d50 0x56092458f0d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.510+0000 7f62ec84e740 1 --2- 172.21.15.53:0/2029275737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560924584430 0x560924584810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.510+0000 7f62ec84e740 1 -- 172.21.15.53:0/2029275737 >> 172.21.15.53:0/2029275737 conn(0x5609245809e0 msgr2=0x560924580dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:45.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.510+0000 7f62ec84e740 1 -- 172.21.15.53:0/2029275737 shutdown_connections 2024-09-19T06:16:45.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.510+0000 7f62ec84e740 1 -- 172.21.15.53:0/2029275737 wait complete. 2024-09-19T06:16:45.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.510+0000 7f62ec84e740 1 -- start start 2024-09-19T06:16:45.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.510+0000 7f62ec84e740 1 Processor -- start 2024-09-19T06:16:45.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.511+0000 7f62ec84e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560924584430 0x56092458f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:45.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.511+0000 7f62ec84e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560924584d50 0x56092458fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:45.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.511+0000 7f62ec84e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560924484ac0 con 0x560924584d50 2024-09-19T06:16:45.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.511+0000 7f62ec84e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5609244e6140 con 0x560924584430 2024-09-19T06:16:45.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.511+0000 7f62ec84e740 10 client.0 _ll_register_callbacks cb 0x5609241f8920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:45.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.511+0000 7f62dbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560924584d50 0x56092458fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:45.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.511+0000 7f62dbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560924584d50 0x56092458fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46040/0 (socket says 172.21.15.53:46040) 2024-09-19T06:16:45.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.511+0000 7f62dbfff640 1 -- 172.21.15.53:0/1598519570 learned_addr learned my addr 172.21.15.53:0/1598519570 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:45.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.511+0000 7f62dbfff640 1 -- 172.21.15.53:0/1598519570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560924584430 msgr2=0x56092458f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:45.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.511+0000 7f62dbfff640 1 --2- 172.21.15.53:0/1598519570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560924584430 0x56092458f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.511+0000 7f62dbfff640 1 -- 172.21.15.53:0/1598519570 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609245903a0 con 0x560924584d50 2024-09-19T06:16:45.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.511+0000 7f62dbfff640 1 --2- 172.21.15.53:0/1598519570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560924584d50 0x56092458fe60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f62cc01dd30 tx=0x7f62cc01dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f92bcd0570b0076 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:45.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.511+0000 7f62c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-19T06:16:45.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.511+0000 7f62c37fe640 1 -- 172.21.15.53:0/1598519570 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f62cc035060 con 0x560924584d50 2024-09-19T06:16:45.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.511+0000 7f62c37fe640 1 -- 172.21.15.53:0/1598519570 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 7138+0+0 (secure 0 0 0) 0x7f62cc02c030 con 0x560924584d50 2024-09-19T06:16:45.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.511+0000 7f62ec84e740 1 -- 172.21.15.53:0/1598519570 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62cc01d900 con 0x560924584d50 2024-09-19T06:16:45.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.512+0000 7f62c37fe640 1 -- 172.21.15.53:0/1598519570 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f62cc03b050 con 0x560924584d50 2024-09-19T06:16:45.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.512+0000 7f62ec84e740 10 client.14532 fetch_fsmap learned FSMap version 8 2024-09-19T06:16:45.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.512+0000 7f62ec84e740 1 -- 172.21.15.53:0/1598519570 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=3+}) -- 0x56092459ac20 con 0x560924584d50 2024-09-19T06:16:45.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.512+0000 7f62c37fe640 1 -- 172.21.15.53:0/1598519570 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62cc01d900 con 0x560924584d50 2024-09-19T06:16:45.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.512+0000 7f62c37fe640 1 -- 172.21.15.53:0/1598519570 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 8) ==== 2968+0+0 (secure 0 0 0) 0x7f62cc03e020 con 0x560924584d50 2024-09-19T06:16:45.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.512+0000 7f62ec84e740 10 client.14532 fetch_fsmap finished waiting for FSMap version 8 2024-09-19T06:16:45.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.512+0000 7f62ec84e740 10 client.14532 resolve_mds: resolved name 'foo.smithi053.rdifzk' to daemon mds.foo.smithi053.rdifzk 2024-09-19T06:16:45.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.512+0000 7f62ec84e740 20 client.14532 populate_metadata read hostname 'smithi053' 2024-09-19T06:16:45.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.512+0000 7f62ec84e740 1 --2- 172.21.15.53:0/1598519570 >> [v2:172.21.15.53:6834/2573849324,v1:172.21.15.53:6835/2573849324] conn(0x56092459cc10 0x56092459f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:45.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.512+0000 7f62ec84e740 4 client.14532 mds_command: new command op to 14510 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:16:45.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.512+0000 7f62ec84e740 1 -- 172.21.15.53:0/1598519570 --> [v2:172.21.15.53:6834/2573849324,v1:172.21.15.53:6835/2573849324] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609244f76f0 con 0x56092459cc10 2024-09-19T06:16:45.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.512+0000 7f62e3fff640 1 --2- 172.21.15.53:0/1598519570 >> [v2:172.21.15.53:6834/2573849324,v1:172.21.15.53:6835/2573849324] conn(0x56092459cc10 0x56092459f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:45.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.512+0000 7f62c37fe640 10 client.14532 ms_handle_connect on v2:172.21.15.53:6834/2573849324 2024-09-19T06:16:45.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.512+0000 7f62e3fff640 1 --2- 172.21.15.53:0/1598519570 >> [v2:172.21.15.53:6834/2573849324,v1:172.21.15.53:6835/2573849324] conn(0x56092459cc10 0x56092459f000 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d7d7ea13e139f16 server_cookie=503beb5a5ac2b08f in_seq=0 out_seq=0 2024-09-19T06:16:45.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.513+0000 7f62c37fe640 1 -- 172.21.15.53:0/1598519570 <== mds.0 v2:172.21.15.53:6834/2573849324 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5609244f76f0 con 0x56092459cc10 2024-09-19T06:16:45.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.513+0000 7f62c37fe640 10 client.14532 handle_command_reply: tid=0 2024-09-19T06:16:45.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.513+0000 7f62ec84e740 1 client.14532 shutdown 2024-09-19T06:16:45.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.513+0000 7f62ec84e740 1 -- 172.21.15.53:0/1598519570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560924584d50 msgr2=0x56092458fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:45.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.513+0000 7f62ec84e740 1 --2- 172.21.15.53:0/1598519570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560924584d50 0x56092458fe60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f62cc01dd30 tx=0x7f62cc01dd60 comp rx=0 tx=0).stop 2024-09-19T06:16:45.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.513+0000 7f62ec84e740 1 -- 172.21.15.53:0/1598519570 shutdown_connections 2024-09-19T06:16:45.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.513+0000 7f62ec84e740 1 --2- 172.21.15.53:0/1598519570 >> [v2:172.21.15.53:6834/2573849324,v1:172.21.15.53:6835/2573849324] conn(0x56092459cc10 0x56092459f000 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.513+0000 7f62ec84e740 1 --2- 172.21.15.53:0/1598519570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560924584d50 0x56092458fe60 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.513+0000 7f62ec84e740 1 --2- 172.21.15.53:0/1598519570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560924584430 0x56092458f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.513+0000 7f62c37fe640 0 client.14532 ms_handle_reset on v2:172.21.15.53:6834/2573849324 2024-09-19T06:16:45.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.513+0000 7f62ec84e740 1 -- 172.21.15.53:0/1598519570 >> 172.21.15.53:0/1598519570 conn(0x5609245809e0 msgr2=0x560924586720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:45.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.513+0000 7f62ec84e740 1 -- 172.21.15.53:0/1598519570 shutdown_connections 2024-09-19T06:16:45.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.513+0000 7f62ec84e740 1 -- 172.21.15.53:0/1598519570 wait complete. 2024-09-19T06:16:45.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.514+0000 7f62ec84e740 20 client.14532 trim_cache size 0 max 16384 2024-09-19T06:16:45.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.532+0000 7f62ec84e740 1 Processor -- start 2024-09-19T06:16:45.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.532+0000 7f62ec84e740 1 -- start start 2024-09-19T06:16:45.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.532+0000 7f62ec84e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560924584430 0x5609245865a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:45.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.533+0000 7f62ec84e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560924584d50 0x5609245a4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:45.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.533+0000 7f62ec84e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f62e41a87f0 con 0x560924584d50 2024-09-19T06:16:45.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.533+0000 7f62ec84e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f62e41a97f0 con 0x560924584430 2024-09-19T06:16:45.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.533+0000 7f62dbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560924584d50 0x5609245a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:45.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.533+0000 7f62dbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560924584d50 0x5609245a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46052/0 (socket says 172.21.15.53:46052) 2024-09-19T06:16:45.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.533+0000 7f62dbfff640 1 -- 172.21.15.53:0/4132763722 learned_addr learned my addr 172.21.15.53:0/4132763722 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:45.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.533+0000 7f62e3fff640 1 --2- 172.21.15.53:0/4132763722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560924584430 0x5609245865a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:45.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.533+0000 7f62dbfff640 1 -- 172.21.15.53:0/4132763722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560924584430 msgr2=0x5609245865a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:45.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.533+0000 7f62dbfff640 1 --2- 172.21.15.53:0/4132763722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560924584430 0x5609245865a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.533+0000 7f62dbfff640 1 -- 172.21.15.53:0/4132763722 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56092425b420 con 0x560924584d50 2024-09-19T06:16:45.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.533+0000 7f62dbfff640 1 --2- 172.21.15.53:0/4132763722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560924584d50 0x5609245a4600 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0268e0 tx=0x7f62cc00af50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b8006c516b6ff8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:45.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.533+0000 7f62d9ffb640 1 -- 172.21.15.53:0/4132763722 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f62cc044070 con 0x560924584d50 2024-09-19T06:16:45.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.533+0000 7f62d9ffb640 1 -- 172.21.15.53:0/4132763722 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62cc036e50 con 0x560924584d50 2024-09-19T06:16:45.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.533+0000 7f62d9ffb640 1 -- 172.21.15.53:0/4132763722 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f62cc014040 con 0x560924584d50 2024-09-19T06:16:45.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.534+0000 7f62ec84e740 1 -- 172.21.15.53:0/4132763722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560924584d50 msgr2=0x5609245a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:45.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.534+0000 7f62ec84e740 1 --2- 172.21.15.53:0/4132763722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560924584d50 0x5609245a4600 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0268e0 tx=0x7f62cc00af50 comp rx=0 tx=0).stop 2024-09-19T06:16:45.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.534+0000 7f62ec84e740 1 -- 172.21.15.53:0/4132763722 shutdown_connections 2024-09-19T06:16:45.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.534+0000 7f62ec84e740 1 --2- 172.21.15.53:0/4132763722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560924584d50 0x5609245a4600 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.534+0000 7f62ec84e740 1 --2- 172.21.15.53:0/4132763722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560924584430 0x5609245865a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.534+0000 7f62ec84e740 1 -- 172.21.15.53:0/4132763722 >> 172.21.15.53:0/4132763722 conn(0x5609245809e0 msgr2=0x560924585e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:45.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.534+0000 7f62ec84e740 1 -- 172.21.15.53:0/4132763722 shutdown_connections 2024-09-19T06:16:45.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.534+0000 7f62ec84e740 1 -- 172.21.15.53:0/4132763722 wait complete. 2024-09-19T06:16:45.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.535+0000 7f62ec84e740 1 -- start start 2024-09-19T06:16:45.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.535+0000 7f62ec84e740 1 Processor -- start 2024-09-19T06:16:45.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.535+0000 7f62ec84e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560924584430 0x560924595d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:45.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.535+0000 7f62ec84e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560924584d50 0x5609245930c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:45.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.535+0000 7f62ec84e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56092459b3e0 con 0x560924584430 2024-09-19T06:16:45.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.535+0000 7f62ec84e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56092458f610 con 0x560924584d50 2024-09-19T06:16:45.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.535+0000 7f62ec84e740 10 client.0 _ll_register_callbacks cb 0x7f62e406b210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:45.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.535+0000 7f62e3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560924584430 0x560924595d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:45.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.535+0000 7f62e3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560924584430 0x560924595d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46064/0 (socket says 172.21.15.53:46064) 2024-09-19T06:16:45.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.535+0000 7f62e3fff640 1 -- 172.21.15.53:0/3019568573 learned_addr learned my addr 172.21.15.53:0/3019568573 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:45.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.535+0000 7f62dbfff640 1 --2- 172.21.15.53:0/3019568573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560924584d50 0x5609245930c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:45.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.536+0000 7f62e3fff640 1 -- 172.21.15.53:0/3019568573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560924584d50 msgr2=0x5609245930c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:45.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.536+0000 7f62e3fff640 1 --2- 172.21.15.53:0/3019568573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560924584d50 0x5609245930c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.536+0000 7f62e3fff640 1 -- 172.21.15.53:0/3019568573 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560924580810 con 0x560924584430 2024-09-19T06:16:45.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.536+0000 7f62e3fff640 1 --2- 172.21.15.53:0/3019568573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560924584430 0x560924595d70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f62d40845a0 tx=0x7f62d4004d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e21ea8c10d412ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:45.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.536+0000 7f62c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-19T06:16:45.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.536+0000 7f62c37fe640 1 -- 172.21.15.53:0/3019568573 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f62d4049040 con 0x560924584430 2024-09-19T06:16:45.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.536+0000 7f62c37fe640 1 -- 172.21.15.53:0/3019568573 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 7138+0+0 (secure 0 0 0) 0x7f62d40af0d0 con 0x560924584430 2024-09-19T06:16:45.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.536+0000 7f62ec84e740 1 -- 172.21.15.53:0/3019568573 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62cc036e50 con 0x560924584430 2024-09-19T06:16:45.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.536+0000 7f62c37fe640 1 -- 172.21.15.53:0/3019568573 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f62d408b050 con 0x560924584430 2024-09-19T06:16:45.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.536+0000 7f62ec84e740 10 client.14536 fetch_fsmap learned FSMap version 8 2024-09-19T06:16:45.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.536+0000 7f62ec84e740 1 -- 172.21.15.53:0/3019568573 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=3+}) -- 0x560924606ac0 con 0x560924584430 2024-09-19T06:16:45.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.536+0000 7f62c37fe640 1 -- 172.21.15.53:0/3019568573 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62cc036e50 con 0x560924584430 2024-09-19T06:16:45.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.537+0000 7f62c37fe640 1 -- 172.21.15.53:0/3019568573 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 8) ==== 2968+0+0 (secure 0 0 0) 0x7f62d4078560 con 0x560924584430 2024-09-19T06:16:45.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.537+0000 7f62ec84e740 10 client.14536 fetch_fsmap finished waiting for FSMap version 8 2024-09-19T06:16:45.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.537+0000 7f62ec84e740 10 client.14536 resolve_mds: resolved name 'foo.smithi053.rdifzk' to daemon mds.foo.smithi053.rdifzk 2024-09-19T06:16:45.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.537+0000 7f62ec84e740 20 client.14536 populate_metadata read hostname 'smithi053' 2024-09-19T06:16:45.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.537+0000 7f62ec84e740 1 --2- 172.21.15.53:0/3019568573 >> [v2:172.21.15.53:6834/2573849324,v1:172.21.15.53:6835/2573849324] conn(0x560924607a80 0x560924609ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:45.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.537+0000 7f62ec84e740 4 client.14536 mds_command: new command op to 14510 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:16:45.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.537+0000 7f62ec84e740 1 -- 172.21.15.53:0/3019568573 --> [v2:172.21.15.53:6834/2573849324,v1:172.21.15.53:6835/2573849324] -- command(tid 0: {"prefix": "status"}) -- 0x5609244f1c00 con 0x560924607a80 2024-09-19T06:16:45.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.537+0000 7f62dbfff640 1 --2- 172.21.15.53:0/3019568573 >> [v2:172.21.15.53:6834/2573849324,v1:172.21.15.53:6835/2573849324] conn(0x560924607a80 0x560924609ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:45.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.537+0000 7f62c37fe640 10 client.14536 ms_handle_connect on v2:172.21.15.53:6834/2573849324 2024-09-19T06:16:45.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.537+0000 7f62dbfff640 1 --2- 172.21.15.53:0/3019568573 >> [v2:172.21.15.53:6834/2573849324,v1:172.21.15.53:6835/2573849324] conn(0x560924607a80 0x560924609ef0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fff2ac9e2a7ed9fe server_cookie=f0ac39ea336f44a5 in_seq=0 out_seq=0 2024-09-19T06:16:45.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.538+0000 7f62c37fe640 1 -- 172.21.15.53:0/3019568573 <== mds.0 v2:172.21.15.53:6834/2573849324 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x5609244f1c00 con 0x560924607a80 2024-09-19T06:16:45.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.538+0000 7f62c37fe640 10 client.14536 handle_command_reply: tid=0 2024-09-19T06:16:45.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.538+0000 7f62ec84e740 1 client.14536 shutdown 2024-09-19T06:16:45.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.538+0000 7f62ec84e740 1 -- 172.21.15.53:0/3019568573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560924584430 msgr2=0x560924595d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:45.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.538+0000 7f62ec84e740 1 --2- 172.21.15.53:0/3019568573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560924584430 0x560924595d70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f62d40845a0 tx=0x7f62d4004d80 comp rx=0 tx=0).stop 2024-09-19T06:16:45.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.538+0000 7f62ec84e740 1 -- 172.21.15.53:0/3019568573 shutdown_connections 2024-09-19T06:16:45.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.538+0000 7f62ec84e740 1 --2- 172.21.15.53:0/3019568573 >> [v2:172.21.15.53:6834/2573849324,v1:172.21.15.53:6835/2573849324] conn(0x560924607a80 0x560924609ef0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.538+0000 7f62ec84e740 1 --2- 172.21.15.53:0/3019568573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560924584d50 0x5609245930c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.538+0000 7f62ec84e740 1 --2- 172.21.15.53:0/3019568573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560924584430 0x560924595d70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.538+0000 7f62ec84e740 1 -- 172.21.15.53:0/3019568573 >> 172.21.15.53:0/3019568573 conn(0x5609245809e0 msgr2=0x5609245865e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:45.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.538+0000 7f62c37fe640 0 client.14536 ms_handle_reset on v2:172.21.15.53:6834/2573849324 2024-09-19T06:16:45.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.538+0000 7f62ec84e740 1 -- 172.21.15.53:0/3019568573 shutdown_connections 2024-09-19T06:16:45.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.538+0000 7f62ec84e740 1 -- 172.21.15.53:0/3019568573 wait complete. 2024-09-19T06:16:45.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.538+0000 7f62ec84e740 20 client.14536 trim_cache size 0 max 16384 2024-09-19T06:16:45.544 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-19T06:16:45.544 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "9f2876d0-764d-11ef-bceb-c7b262605968", 2024-09-19T06:16:45.544 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2024-09-19T06:16:45.544 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 14510, 2024-09-19T06:16:45.544 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2024-09-19T06:16:45.544 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2024-09-19T06:16:45.544 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "foo", 2024-09-19T06:16:45.544 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 8.7393877050000004, 2024-09-19T06:16:45.544 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 8, 2024-09-19T06:16:45.545 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 69, 2024-09-19T06:16:45.545 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 69, 2024-09-19T06:16:45.545 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 9.2153424360000002 2024-09-19T06:16:45.545 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-19T06:16:45.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.539+0000 7f62ea475640 1 -- 172.21.15.53:0/1256177023 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f62d006e310 msgr2=0x7f62d00707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:45.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.539+0000 7f62ea475640 1 --2- 172.21.15.53:0/1256177023 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f62d006e310 0x7f62d00707d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f62e4198f30 tx=0x7f62cc005ec0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.539+0000 7f62ea475640 1 -- 172.21.15.53:0/1256177023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62e4102ac0 msgr2=0x7f62e419d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:45.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.539+0000 7f62ea475640 1 --2- 172.21.15.53:0/1256177023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62e4102ac0 0x7f62e419d610 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f62d4006010 tx=0x7f62d40048f0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.539+0000 7f62ea475640 1 -- 172.21.15.53:0/1256177023 shutdown_connections 2024-09-19T06:16:45.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.539+0000 7f62ea475640 1 --2- 172.21.15.53:0/1256177023 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f62d006e310 0x7f62d00707d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.539+0000 7f62ea475640 1 --2- 172.21.15.53:0/1256177023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62e41033e0 0x7f62e419db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.539+0000 7f62ea475640 1 --2- 172.21.15.53:0/1256177023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62e4102ac0 0x7f62e419d610 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.539+0000 7f62ea475640 1 -- 172.21.15.53:0/1256177023 >> 172.21.15.53:0/1256177023 conn(0x7f62e40fc520 msgr2=0x7f62e4108f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:45.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.539+0000 7f62ea475640 1 -- 172.21.15.53:0/1256177023 shutdown_connections 2024-09-19T06:16:45.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:45.539+0000 7f62ea475640 1 -- 172.21.15.53:0/1256177023 wait complete. 2024-09-19T06:16:46.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:45 smithi053 ceph-mon[27968]: pgmap v215: 129 pgs: 129 active+clean; 581 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s wr, 11 op/s 2024-09-19T06:16:46.218 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:16:46.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:45 smithi155 ceph-mon[35292]: pgmap v215: 129 pgs: 129 active+clean; 581 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s wr, 11 op/s 2024-09-19T06:16:46.493 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:46.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.914+0000 7f7893076640 1 -- 172.21.15.53:0/311894115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f788c106800 msgr2=0x7f788c106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:46.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.914+0000 7f7893076640 1 --2- 172.21.15.53:0/311894115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f788c106800 0x7f788c106be0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7880009900 tx=0x7f788002c6e0 comp rx=0 tx=0).stop 2024-09-19T06:16:46.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.914+0000 7f7893076640 1 -- 172.21.15.53:0/311894115 shutdown_connections 2024-09-19T06:16:46.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.914+0000 7f7893076640 1 --2- 172.21.15.53:0/311894115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f788c100800 0x7f788c100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:46.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.914+0000 7f7893076640 1 --2- 172.21.15.53:0/311894115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f788c106800 0x7f788c106be0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:46.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.914+0000 7f7893076640 1 -- 172.21.15.53:0/311894115 >> 172.21.15.53:0/311894115 conn(0x7f788c0fc520 msgr2=0x7f788c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:46.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.914+0000 7f7893076640 1 -- 172.21.15.53:0/311894115 shutdown_connections 2024-09-19T06:16:46.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.914+0000 7f7893076640 1 -- 172.21.15.53:0/311894115 wait complete. 2024-09-19T06:16:46.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.914+0000 7f7893076640 1 Processor -- start 2024-09-19T06:16:46.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.915+0000 7f7893076640 1 -- start start 2024-09-19T06:16:46.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.915+0000 7f7893076640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f788c100800 0x7f788c195700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:46.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.915+0000 7f7893076640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f788c106800 0x7f788c195c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:46.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.915+0000 7f7893076640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f788c10cb50 con 0x7f788c100800 2024-09-19T06:16:46.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.915+0000 7f7893076640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f788c10c9d0 con 0x7f788c106800 2024-09-19T06:16:46.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.915+0000 7f7890e12640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f788c100800 0x7f788c195700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:46.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.915+0000 7f7890e12640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f788c100800 0x7f788c195700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46082/0 (socket says 172.21.15.53:46082) 2024-09-19T06:16:46.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.915+0000 7f7890e12640 1 -- 172.21.15.53:0/2898731900 learned_addr learned my addr 172.21.15.53:0/2898731900 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:46.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.915+0000 7f788bfff640 1 --2- 172.21.15.53:0/2898731900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f788c106800 0x7f788c195c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:46.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.915+0000 7f7890e12640 1 -- 172.21.15.53:0/2898731900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f788c106800 msgr2=0x7f788c195c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:46.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.915+0000 7f7890e12640 1 --2- 172.21.15.53:0/2898731900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f788c106800 0x7f788c195c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:46.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.915+0000 7f7890e12640 1 -- 172.21.15.53:0/2898731900 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f788c1962f0 con 0x7f788c100800 2024-09-19T06:16:46.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.916+0000 7f7890e12640 1 --2- 172.21.15.53:0/2898731900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f788c100800 0x7f788c195700 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f788002cc00 tx=0x7f78800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:46.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.916+0000 7f7889ffb640 1 -- 172.21.15.53:0/2898731900 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7880041070 con 0x7f788c100800 2024-09-19T06:16:46.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.916+0000 7f7889ffb640 1 -- 172.21.15.53:0/2898731900 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7880004ac0 con 0x7f788c100800 2024-09-19T06:16:46.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.916+0000 7f7893076640 1 -- 172.21.15.53:0/2898731900 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f788c19a030 con 0x7f788c100800 2024-09-19T06:16:46.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.917+0000 7f7889ffb640 1 -- 172.21.15.53:0/2898731900 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7880037030 con 0x7f788c100800 2024-09-19T06:16:47.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.917+0000 7f7893076640 1 -- 172.21.15.53:0/2898731900 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f788c19a560 con 0x7f788c100800 2024-09-19T06:16:47.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.918+0000 7f7893076640 1 -- 172.21.15.53:0/2898731900 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f788c101f40 con 0x7f788c100800 2024-09-19T06:16:47.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.918+0000 7f7889ffb640 1 -- 172.21.15.53:0/2898731900 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f7880044070 con 0x7f788c100800 2024-09-19T06:16:47.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.919+0000 7f7889ffb640 1 --2- 172.21.15.53:0/2898731900 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f787c06e350 0x7f787c070810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:47.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.919+0000 7f788bfff640 1 --2- 172.21.15.53:0/2898731900 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f787c06e350 0x7f787c070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:47.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.919+0000 7f7889ffb640 1 -- 172.21.15.53:0/2898731900 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7138+0+0 (secure 0 0 0) 0x7f78800b1170 con 0x7f788c100800 2024-09-19T06:16:47.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.919+0000 7f788bfff640 1 --2- 172.21.15.53:0/2898731900 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f787c06e350 0x7f787c070810 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7878005fd0 tx=0x7f7878005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:47.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:46.924+0000 7f7889ffb640 1 -- 172.21.15.53:0/2898731900 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f788007e4d0 con 0x7f788c100800 2024-09-19T06:16:47.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:47.154+0000 7f7893076640 1 -- 172.21.15.53:0/2898731900 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f788c108d70 con 0x7f788c100800 2024-09-19T06:16:47.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:47.154+0000 7f7889ffb640 1 -- 172.21.15.53:0/2898731900 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+4923 (secure 0 0 0) 0x7f7880082180 con 0x7f788c100800 2024-09-19T06:16:47.157 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:16:47.157 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":8,"btime":"2024-09-19T06:16:41:472413+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T06:16:32.878153+0000","modified":"2024-09-19T06:16:40.819179+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":14510},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14510":{"gid":14510,"name":"foo.smithi053.rdifzk","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/2573849324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2573849324},{"type":"v1","addr":"172.21.15.53:6835","nonce":2573849324}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"foo","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14510,"qdb_cluster":[14510]},"id":1},{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T06:16:36.000321+0000","modified":"2024-09-19T06:16:40.470997+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24319":{"gid":24319,"name":"foo.smithi155.dsbavw","rank":0,"incarnation":6,"state":"up:active","state_seq":2,"addr":"172.21.15.155:6833/390026217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":390026217},{"type":"v1","addr":"172.21.15.155:6833","nonce":390026217}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"bar","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24319,"qdb_cluster":[24319]},"id":2}]} 2024-09-19T06:16:47.157 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 8 2024-09-19T06:16:47.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:47.157+0000 7f7893076640 1 -- 172.21.15.53:0/2898731900 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f787c06e350 msgr2=0x7f787c070810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:47.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:47.157+0000 7f7893076640 1 --2- 172.21.15.53:0/2898731900 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f787c06e350 0x7f787c070810 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7878005fd0 tx=0x7f7878005850 comp rx=0 tx=0).stop 2024-09-19T06:16:47.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:47.157+0000 7f7893076640 1 -- 172.21.15.53:0/2898731900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f788c100800 msgr2=0x7f788c195700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:47.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:47.157+0000 7f7893076640 1 --2- 172.21.15.53:0/2898731900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f788c100800 0x7f788c195700 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f788002cc00 tx=0x7f78800046c0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:47.157+0000 7f7893076640 1 -- 172.21.15.53:0/2898731900 shutdown_connections 2024-09-19T06:16:47.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:47.157+0000 7f7893076640 1 --2- 172.21.15.53:0/2898731900 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f787c06e350 0x7f787c070810 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:47.157+0000 7f7893076640 1 --2- 172.21.15.53:0/2898731900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f788c106800 0x7f788c195c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:47.157+0000 7f7893076640 1 --2- 172.21.15.53:0/2898731900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f788c100800 0x7f788c195700 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:47.157+0000 7f7893076640 1 -- 172.21.15.53:0/2898731900 >> 172.21.15.53:0/2898731900 conn(0x7f788c0fc520 msgr2=0x7f788c0fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:47.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:47.157+0000 7f7893076640 1 -- 172.21.15.53:0/2898731900 shutdown_connections 2024-09-19T06:16:47.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:47.157+0000 7f7893076640 1 -- 172.21.15.53:0/2898731900 wait complete. 2024-09-19T06:16:47.476 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:47 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:47.476 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:47 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:47.476 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:47 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:16:47.476 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:47 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:16:47.476 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:47 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:47.476 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:47 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.bar.smithi053.nzdkgr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:16:47.476 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:47 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.bar.smithi053.nzdkgr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:16:47.477 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:47 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:16:47.477 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:47 smithi053 ceph-mon[27968]: Deploying daemon mds.bar.smithi053.nzdkgr on smithi053 2024-09-19T06:16:47.477 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:47 smithi053 ceph-mon[27968]: pgmap v216: 129 pgs: 129 active+clean; 581 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s wr, 8 op/s 2024-09-19T06:16:47.477 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:47 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/2898731900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:16:47.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:47 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:47.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:47 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:47.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:47 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:16:47.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:47 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:16:47.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:47 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:47.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:47 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.bar.smithi053.nzdkgr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:16:47.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:47 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.bar.smithi053.nzdkgr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:16:47.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:47 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:16:47.734 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:47 smithi155 ceph-mon[35292]: Deploying daemon mds.bar.smithi053.nzdkgr on smithi053 2024-09-19T06:16:47.734 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:47 smithi155 ceph-mon[35292]: pgmap v216: 129 pgs: 129 active+clean; 581 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s wr, 8 op/s 2024-09-19T06:16:47.734 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:47 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/2898731900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:16:48.221 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:16:48.722 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:48.964 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:48 smithi053 ceph-mon[27968]: pgmap v217: 129 pgs: 129 active+clean; 581 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s wr, 7 op/s 2024-09-19T06:16:48.964 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:48 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:48.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:48 smithi155 ceph-mon[35292]: pgmap v217: 129 pgs: 129 active+clean; 581 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s wr, 7 op/s 2024-09-19T06:16:48.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:48 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:49.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.382+0000 7fadb51df640 1 -- 172.21.15.53:0/3823158708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadb0100800 msgr2=0x7fadb0100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:49.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.382+0000 7fadb51df640 1 --2- 172.21.15.53:0/3823158708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadb0100800 0x7fadb0100c60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fada4009880 tx=0x7fada402a4a0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.383+0000 7fadb51df640 1 -- 172.21.15.53:0/3823158708 shutdown_connections 2024-09-19T06:16:49.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.383+0000 7fadb51df640 1 --2- 172.21.15.53:0/3823158708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadb0100800 0x7fadb0100c60 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.383+0000 7fadb51df640 1 --2- 172.21.15.53:0/3823158708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadb0106800 0x7fadb0106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.383+0000 7fadb51df640 1 -- 172.21.15.53:0/3823158708 >> 172.21.15.53:0/3823158708 conn(0x7fadb00fc520 msgr2=0x7fadb00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:49.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.383+0000 7fadb51df640 1 -- 172.21.15.53:0/3823158708 shutdown_connections 2024-09-19T06:16:49.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.383+0000 7fadb51df640 1 -- 172.21.15.53:0/3823158708 wait complete. 2024-09-19T06:16:49.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.383+0000 7fadb51df640 1 Processor -- start 2024-09-19T06:16:49.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.384+0000 7fadb51df640 1 -- start start 2024-09-19T06:16:49.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.384+0000 7fadb51df640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadb0100800 0x7fadb0198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:49.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.384+0000 7fadb51df640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadb0106800 0x7fadb0198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:49.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.384+0000 7fadb51df640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fadb010dc20 con 0x7fadb0100800 2024-09-19T06:16:49.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.384+0000 7fadb51df640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fadb010daa0 con 0x7fadb0106800 2024-09-19T06:16:49.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.384+0000 7fadaed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadb0100800 0x7fadb0198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:49.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.384+0000 7fadaed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadb0100800 0x7fadb0198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46106/0 (socket says 172.21.15.53:46106) 2024-09-19T06:16:49.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.384+0000 7fadaed9d640 1 -- 172.21.15.53:0/3116291761 learned_addr learned my addr 172.21.15.53:0/3116291761 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:49.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.384+0000 7fadae59c640 1 --2- 172.21.15.53:0/3116291761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadb0106800 0x7fadb0198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:49.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.384+0000 7fadaed9d640 1 -- 172.21.15.53:0/3116291761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadb0106800 msgr2=0x7fadb0198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:49.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.384+0000 7fadaed9d640 1 --2- 172.21.15.53:0/3116291761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadb0106800 0x7fadb0198cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.384+0000 7fadaed9d640 1 -- 172.21.15.53:0/3116291761 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb0199230 con 0x7fadb0100800 2024-09-19T06:16:49.403 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.384+0000 7fadae59c640 1 --2- 172.21.15.53:0/3116291761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadb0106800 0x7fadb0198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:16:49.403 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.384+0000 7fadaed9d640 1 --2- 172.21.15.53:0/3116291761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadb0100800 0x7fadb0198780 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fada000d8d0 tx=0x7fada000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:49.403 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.385+0000 7fad97fff640 1 -- 172.21.15.53:0/3116291761 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fada0017070 con 0x7fadb0100800 2024-09-19T06:16:49.403 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.385+0000 7fad97fff640 1 -- 172.21.15.53:0/3116291761 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fada0010460 con 0x7fadb0100800 2024-09-19T06:16:49.403 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.385+0000 7fadb51df640 1 -- 172.21.15.53:0/3116291761 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadb0199520 con 0x7fadb0100800 2024-09-19T06:16:49.403 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.385+0000 7fad97fff640 1 -- 172.21.15.53:0/3116291761 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fada000f5f0 con 0x7fadb0100800 2024-09-19T06:16:49.403 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.385+0000 7fadb51df640 1 -- 172.21.15.53:0/3116291761 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadb01a1b20 con 0x7fadb0100800 2024-09-19T06:16:49.403 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.386+0000 7fadb51df640 1 -- 172.21.15.53:0/3116291761 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadb0101f40 con 0x7fadb0100800 2024-09-19T06:16:49.403 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.386+0000 7fad97fff640 1 -- 172.21.15.53:0/3116291761 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7fada0010610 con 0x7fadb0100800 2024-09-19T06:16:49.403 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.387+0000 7fad97fff640 1 --2- 172.21.15.53:0/3116291761 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fad9806e2b0 0x7fad98070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:49.403 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.387+0000 7fad97fff640 1 -- 172.21.15.53:0/3116291761 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7138+0+0 (secure 0 0 0) 0x7fada008e6e0 con 0x7fadb0100800 2024-09-19T06:16:49.404 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.387+0000 7fadae59c640 1 --2- 172.21.15.53:0/3116291761 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fad9806e2b0 0x7fad98070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:49.404 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.387+0000 7fadae59c640 1 --2- 172.21.15.53:0/3116291761 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fad9806e2b0 0x7fad98070770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fada4006010 tx=0x7fada40095f0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:49.404 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.391+0000 7fad97fff640 1 -- 172.21.15.53:0/3116291761 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fada005b920 con 0x7fadb0100800 2024-09-19T06:16:49.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.584+0000 7fadb51df640 1 -- 172.21.15.53:0/3116291761 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fadb0100c60 con 0x7fadb0100800 2024-09-19T06:16:49.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.586+0000 7fad97fff640 1 -- 172.21.15.53:0/3116291761 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+21340 (secure 0 0 0) 0x7fada005f5d0 con 0x7fadb0100800 2024-09-19T06:16:49.589 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:16:49.589 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":69,"fsid":"9f2876d0-764d-11ef-bceb-c7b262605968","created":"2024-09-19T06:09:26.797853+0000","modified":"2024-09-19T06:16:38.457146+0000","last_up_change":"2024-09-19T06:15:47.809832+0000","last_in_change":"2024-09-19T06:15:26.420480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T06:13:39.241335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.foo.meta","create_time":"2024-09-19T06:16:31.363649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"foo"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.foo.data","create_time":"2024-09-19T06:16:32.311547+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"foo"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs.bar.meta","create_time":"2024-09-19T06:16:34.984973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"bar"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs.bar.data","create_time":"2024-09-19T06:16:35.435954+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"bar"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"35349d4a-0bde-46f6-8d5f-a019f0f89095","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1456825097},{"type":"v1","addr":"172.21.15.53:6803","nonce":1456825097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1456825097},{"type":"v1","addr":"172.21.15.53:6805","nonce":1456825097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1456825097},{"type":"v1","addr":"172.21.15.53:6809","nonce":1456825097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1456825097},{"type":"v1","addr":"172.21.15.53:6807","nonce":1456825097}]},"public_addr":"172.21.15.53:6803/1456825097","cluster_addr":"172.21.15.53:6805/1456825097","heartbeat_back_addr":"172.21.15.53:6809/1456825097","heartbeat_front_addr":"172.21.15.53:6807/1456825097","state":["exists","up"]},{"osd":1,"uuid":"832c2772-a460-4691-8db7-ad1d0c48cc9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3638097004},{"type":"v1","addr":"172.21.15.53:6811","nonce":3638097004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3638097004},{"type":"v1","addr":"172.21.15.53:6813","nonce":3638097004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3638097004},{"type":"v1","addr":"172.21.15.53:6817","nonce":3638097004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3638097004},{"type":"v1","addr":"172.21.15.53:6815","nonce":3638097004}]},"public_addr":"172.21.15.53:6811/3638097004","cluster_addr":"172.21.15.53:6813/3638097004","heartbeat_back_addr":"172.21.15.53:6817/3638097004","heartbeat_front_addr":"172.21.15.53:6815/3638097004","state":["exists","up"]},{"osd":2,"uuid":"57282117-559f-4495-b85f-90b68b2439ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":1988602192},{"type":"v1","addr":"172.21.15.53:6819","nonce":1988602192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":1988602192},{"type":"v1","addr":"172.21.15.53:6821","nonce":1988602192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1988602192},{"type":"v1","addr":"172.21.15.53:6825","nonce":1988602192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":1988602192},{"type":"v1","addr":"172.21.15.53:6823","nonce":1988602192}]},"public_addr":"172.21.15.53:6819/1988602192","cluster_addr":"172.21.15.53:6821/1988602192","heartbeat_back_addr":"172.21.15.53:6825/1988602192","heartbeat_front_addr":"172.21.15.53:6823/1988602192","state":["exists","up"]},{"osd":3,"uuid":"e6bf8cf0-2419-4c2a-b562-9221a117050c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2604012854},{"type":"v1","addr":"172.21.15.53:6827","nonce":2604012854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2604012854},{"type":"v1","addr":"172.21.15.53:6829","nonce":2604012854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2604012854},{"type":"v1","addr":"172.21.15.53:6833","nonce":2604012854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2604012854},{"type":"v1","addr":"172.21.15.53:6831","nonce":2604012854}]},"public_addr":"172.21.15.53:6827/2604012854","cluster_addr":"172.21.15.53:6829/2604012854","heartbeat_back_addr":"172.21.15.53:6833/2604012854","heartbeat_front_addr":"172.21.15.53:6831/2604012854","state":["exists","up"]},{"osd":4,"uuid":"7adedf0a-6fbc-42bc-8996-ba482b638929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2046029108},{"type":"v1","addr":"172.21.15.155:6801","nonce":2046029108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2046029108},{"type":"v1","addr":"172.21.15.155:6803","nonce":2046029108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2046029108},{"type":"v1","addr":"172.21.15.155:6807","nonce":2046029108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2046029108},{"type":"v1","addr":"172.21.15.155:6805","nonce":2046029108}]},"public_addr":"172.21.15.155:6801/2046029108","cluster_addr":"172.21.15.155:6803/2046029108","heartbeat_back_addr":"172.21.15.155:6807/2046029108","heartbeat_front_addr":"172.21.15.155:6805/2046029108","state":["exists","up"]},{"osd":5,"uuid":"c664e57c-20b5-46d4-8830-70780a91d6dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":464507356},{"type":"v1","addr":"172.21.15.155:6809","nonce":464507356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":464507356},{"type":"v1","addr":"172.21.15.155:6811","nonce":464507356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":464507356},{"type":"v1","addr":"172.21.15.155:6815","nonce":464507356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":464507356},{"type":"v1","addr":"172.21.15.155:6813","nonce":464507356}]},"public_addr":"172.21.15.155:6809/464507356","cluster_addr":"172.21.15.155:6811/464507356","heartbeat_back_addr":"172.21.15.155:6815/464507356","heartbeat_front_addr":"172.21.15.155:6813/464507356","state":["exists","up"]},{"osd":6,"uuid":"bc8a88f5-75ab-4482-88e0-5cc5e6f06db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2574655677},{"type":"v1","addr":"172.21.15.155:6817","nonce":2574655677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2574655677},{"type":"v1","addr":"172.21.15.155:6819","nonce":2574655677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2574655677},{"type":"v1","addr":"172.21.15.155:6823","nonce":2574655677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2574655677},{"type":"v1","addr":"172.21.15.155:6821","nonce":2574655677}]},"public_addr":"172.21.15.155:6817/2574655677","cluster_addr":"172.21.15.155:6819/2574655677","heartbeat_back_addr":"172.21.15.155:6823/2574655677","heartbeat_front_addr":"172.21.15.155:6821/2574655677","state":["exists","up"]},{"osd":7,"uuid":"b91759fe-f27c-4ab2-9bd3-b5961b358d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2269161534},{"type":"v1","addr":"172.21.15.155:6825","nonce":2269161534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2269161534},{"type":"v1","addr":"172.21.15.155:6827","nonce":2269161534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2269161534},{"type":"v1","addr":"172.21.15.155:6831","nonce":2269161534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2269161534},{"type":"v1","addr":"172.21.15.155:6829","nonce":2269161534}]},"public_addr":"172.21.15.155:6825/2269161534","cluster_addr":"172.21.15.155:6827/2269161534","heartbeat_back_addr":"172.21.15.155:6831/2269161534","heartbeat_front_addr":"172.21.15.155:6829/2269161534","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:12:43.760347+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:13:10.493898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:13:37.244856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:14:05.006717+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:14:29.801028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:14:55.095341+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:15:20.683202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:15:45.635704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6801/3684090301":"2024-09-20T06:11:28.721009+0000","172.21.15.53:6800/475342876":"2024-09-20T06:09:58.271775+0000","172.21.15.53:6801/475342876":"2024-09-20T06:09:58.271775+0000","172.21.15.53:0/3753705881":"2024-09-20T06:11:28.721009+0000","172.21.15.53:0/4216592427":"2024-09-20T06:09:58.271775+0000","172.21.15.53:0/1618433917":"2024-09-20T06:10:30.209033+0000","172.21.15.53:0/623598636":"2024-09-20T06:09:58.271775+0000","172.21.15.53:6800/3684090301":"2024-09-20T06:11:28.721009+0000","172.21.15.53:6800/681468213":"2024-09-20T06:10:30.209033+0000","172.21.15.53:0/597749918":"2024-09-20T06:10:30.209033+0000","172.21.15.53:0/21853845":"2024-09-20T06:11:28.721009+0000","172.21.15.53:0/3515303955":"2024-09-20T06:09:58.271775+0000","172.21.15.53:6801/681468213":"2024-09-20T06:10:30.209033+0000","172.21.15.53:0/1729980812":"2024-09-20T06:11:28.721009+0000","172.21.15.53:0/372276981":"2024-09-20T06:10:30.209033+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:16:49.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.588+0000 7fadb51df640 1 -- 172.21.15.53:0/3116291761 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fad9806e2b0 msgr2=0x7fad98070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:49.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.588+0000 7fadb51df640 1 --2- 172.21.15.53:0/3116291761 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fad9806e2b0 0x7fad98070770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fada4006010 tx=0x7fada40095f0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.588+0000 7fadb51df640 1 -- 172.21.15.53:0/3116291761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadb0100800 msgr2=0x7fadb0198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:49.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.588+0000 7fadb51df640 1 --2- 172.21.15.53:0/3116291761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadb0100800 0x7fadb0198780 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fada000d8d0 tx=0x7fada000dd90 comp rx=0 tx=0).stop 2024-09-19T06:16:49.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.589+0000 7fadb51df640 1 -- 172.21.15.53:0/3116291761 shutdown_connections 2024-09-19T06:16:49.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.589+0000 7fadb51df640 1 --2- 172.21.15.53:0/3116291761 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fad9806e2b0 0x7fad98070770 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.589+0000 7fadb51df640 1 --2- 172.21.15.53:0/3116291761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadb0106800 0x7fadb0198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.589+0000 7fadb51df640 1 --2- 172.21.15.53:0/3116291761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadb0100800 0x7fadb0198780 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.589+0000 7fadb51df640 1 -- 172.21.15.53:0/3116291761 >> 172.21.15.53:0/3116291761 conn(0x7fadb00fc520 msgr2=0x7fadb00fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:49.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.589+0000 7fadb51df640 1 -- 172.21.15.53:0/3116291761 shutdown_connections 2024-09-19T06:16:49.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:49.589+0000 7fadb51df640 1 -- 172.21.15.53:0/3116291761 wait complete. 2024-09-19T06:16:50.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:49 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/3116291761' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:16:50.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:49 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/3116291761' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:16:50.571 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 3, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T06:16:36.000321+0000', 'modified': '2024-09-19T06:16:36.000321+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [], 'up': {}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'bar', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-19T06:16:50.571 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-09-19T06:16:51.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:51 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:51.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:51 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:51.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:51 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:51.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:51 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.bar.smithi155.foldmc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:16:51.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:51 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.bar.smithi155.foldmc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:16:51.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:51 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:16:51.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:51 smithi155 ceph-mon[35292]: Deploying daemon mds.bar.smithi155.foldmc on smithi155 2024-09-19T06:16:51.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:51 smithi155 ceph-mon[35292]: pgmap v218: 129 pgs: 129 active+clean; 581 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-19T06:16:51.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:51 smithi155 ceph-mon[35292]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-09-19T06:16:51.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:51 smithi155 ceph-mon[35292]: Cluster is now healthy 2024-09-19T06:16:51.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:51 smithi155 ceph-mon[35292]: mds.? [v2:172.21.15.53:6836/4281289831,v1:172.21.15.53:6837/4281289831] up:boot 2024-09-19T06:16:51.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:51 smithi155 ceph-mon[35292]: Dropping low affinity active daemon mds.foo.smithi155.dsbavw in favor of higher affinity standby. 2024-09-19T06:16:51.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:51 smithi155 ceph-mon[35292]: Replacing daemon mds.foo.smithi155.dsbavw as rank 0 with standby daemon mds.bar.smithi053.nzdkgr 2024-09-19T06:16:51.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:51 smithi155 ceph-mon[35292]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T06:16:51.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:51 smithi155 ceph-mon[35292]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-09-19T06:16:51.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:51 smithi155 ceph-mon[35292]: fsmap foo:1 bar:1 {bar:0=foo.smithi155.dsbavw=up:active,foo:0=foo.smithi053.rdifzk=up:active} 1 up:standby 2024-09-19T06:16:51.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:51 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mds metadata", "who": "bar.smithi053.nzdkgr"}]: dispatch 2024-09-19T06:16:51.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:51 smithi155 ceph-mon[35292]: osdmap e70: 8 total, 8 up, 8 in 2024-09-19T06:16:51.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:51 smithi155 ceph-mon[35292]: fsmap foo:1 bar:1/1 {bar:0=bar.smithi053.nzdkgr=up:replay,foo:0=foo.smithi053.rdifzk=up:active} 2024-09-19T06:16:51.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:51 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:51.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:51 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:51.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:51 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:51.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:51 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.bar.smithi155.foldmc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:16:51.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:51 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.bar.smithi155.foldmc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:16:51.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:51 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:16:51.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:51 smithi053 ceph-mon[27968]: Deploying daemon mds.bar.smithi155.foldmc on smithi155 2024-09-19T06:16:51.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:51 smithi053 ceph-mon[27968]: pgmap v218: 129 pgs: 129 active+clean; 581 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-19T06:16:51.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:51 smithi053 ceph-mon[27968]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-09-19T06:16:51.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:51 smithi053 ceph-mon[27968]: Cluster is now healthy 2024-09-19T06:16:51.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:51 smithi053 ceph-mon[27968]: mds.? [v2:172.21.15.53:6836/4281289831,v1:172.21.15.53:6837/4281289831] up:boot 2024-09-19T06:16:51.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:51 smithi053 ceph-mon[27968]: Dropping low affinity active daemon mds.foo.smithi155.dsbavw in favor of higher affinity standby. 2024-09-19T06:16:51.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:51 smithi053 ceph-mon[27968]: Replacing daemon mds.foo.smithi155.dsbavw as rank 0 with standby daemon mds.bar.smithi053.nzdkgr 2024-09-19T06:16:51.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:51 smithi053 ceph-mon[27968]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T06:16:51.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:51 smithi053 ceph-mon[27968]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-09-19T06:16:51.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:51 smithi053 ceph-mon[27968]: fsmap foo:1 bar:1 {bar:0=foo.smithi155.dsbavw=up:active,foo:0=foo.smithi053.rdifzk=up:active} 1 up:standby 2024-09-19T06:16:51.357 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:51 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mds metadata", "who": "bar.smithi053.nzdkgr"}]: dispatch 2024-09-19T06:16:51.357 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:51 smithi053 ceph-mon[27968]: osdmap e70: 8 total, 8 up, 8 in 2024-09-19T06:16:51.357 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:51 smithi053 ceph-mon[27968]: fsmap foo:1 bar:1/1 {bar:0=bar.smithi053.nzdkgr=up:replay,foo:0=foo.smithi053.rdifzk=up:active} 2024-09-19T06:16:51.573 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:16:51.853 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:52.212 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:52 smithi155 ceph-mon[35292]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-09-19T06:16:52.212 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:52 smithi155 ceph-mon[35292]: mds.? [v2:172.21.15.53:6836/4281289831,v1:172.21.15.53:6837/4281289831] up:reconnect 2024-09-19T06:16:52.212 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:52 smithi155 ceph-mon[35292]: mds.? [v2:172.21.15.155:6832/669003188,v1:172.21.15.155:6833/669003188] up:boot 2024-09-19T06:16:52.212 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:52 smithi155 ceph-mon[35292]: fsmap foo:1 bar:1/1 {bar:0=bar.smithi053.nzdkgr=up:reconnect,foo:0=foo.smithi053.rdifzk=up:active} 1 up:standby 2024-09-19T06:16:52.212 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:52 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mds metadata", "who": "foo.smithi155.dsbavw"}]: dispatch 2024-09-19T06:16:52.253 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:52 smithi053 ceph-mon[27968]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-09-19T06:16:52.253 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:52 smithi053 ceph-mon[27968]: mds.? [v2:172.21.15.53:6836/4281289831,v1:172.21.15.53:6837/4281289831] up:reconnect 2024-09-19T06:16:52.254 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:52 smithi053 ceph-mon[27968]: mds.? [v2:172.21.15.155:6832/669003188,v1:172.21.15.155:6833/669003188] up:boot 2024-09-19T06:16:52.254 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:52 smithi053 ceph-mon[27968]: fsmap foo:1 bar:1/1 {bar:0=bar.smithi053.nzdkgr=up:reconnect,foo:0=foo.smithi053.rdifzk=up:active} 1 up:standby 2024-09-19T06:16:52.254 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:52 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mds metadata", "who": "foo.smithi155.dsbavw"}]: dispatch 2024-09-19T06:16:52.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.251+0000 7fa4703a2640 1 -- 172.21.15.53:0/2558338646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa468106810 msgr2=0x7fa468106bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:52.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.251+0000 7fa4703a2640 1 --2- 172.21.15.53:0/2558338646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa468106810 0x7fa468106bf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa464009880 tx=0x7fa46402a4a0 comp rx=0 tx=0).stop 2024-09-19T06:16:52.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.251+0000 7fa4703a2640 1 -- 172.21.15.53:0/2558338646 shutdown_connections 2024-09-19T06:16:52.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.251+0000 7fa4703a2640 1 --2- 172.21.15.53:0/2558338646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa468100810 0x7fa468100c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:52.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.251+0000 7fa4703a2640 1 --2- 172.21.15.53:0/2558338646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa468106810 0x7fa468106bf0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:52.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.251+0000 7fa4703a2640 1 -- 172.21.15.53:0/2558338646 >> 172.21.15.53:0/2558338646 conn(0x7fa4680fc510 msgr2=0x7fa4680fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:52.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.251+0000 7fa4703a2640 1 -- 172.21.15.53:0/2558338646 shutdown_connections 2024-09-19T06:16:52.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.251+0000 7fa4703a2640 1 -- 172.21.15.53:0/2558338646 wait complete. 2024-09-19T06:16:52.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.252+0000 7fa4703a2640 1 Processor -- start 2024-09-19T06:16:52.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.252+0000 7fa4703a2640 1 -- start start 2024-09-19T06:16:52.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.252+0000 7fa4703a2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa468100810 0x7fa468198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:52.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.252+0000 7fa4703a2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa468106810 0x7fa468198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:52.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.252+0000 7fa4703a2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa46810dc50 con 0x7fa468106810 2024-09-19T06:16:52.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.252+0000 7fa4703a2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa46810dad0 con 0x7fa468100810 2024-09-19T06:16:52.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.252+0000 7fa46d93d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa468106810 0x7fa468198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:52.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.252+0000 7fa46d93d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa468106810 0x7fa468198cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54180/0 (socket says 172.21.15.53:54180) 2024-09-19T06:16:52.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.252+0000 7fa46d93d640 1 -- 172.21.15.53:0/823827149 learned_addr learned my addr 172.21.15.53:0/823827149 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:52.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.252+0000 7fa46e13e640 1 --2- 172.21.15.53:0/823827149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa468100810 0x7fa468198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:52.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.252+0000 7fa46d93d640 1 -- 172.21.15.53:0/823827149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa468100810 msgr2=0x7fa468198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:52.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.252+0000 7fa46d93d640 1 --2- 172.21.15.53:0/823827149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa468100810 0x7fa468198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:52.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.252+0000 7fa46d93d640 1 -- 172.21.15.53:0/823827149 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa468199240 con 0x7fa468106810 2024-09-19T06:16:52.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.252+0000 7fa46e13e640 1 --2- 172.21.15.53:0/823827149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa468100810 0x7fa468198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:16:52.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.253+0000 7fa46d93d640 1 --2- 172.21.15.53:0/823827149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa468106810 0x7fa468198cd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa45800d6f0 tx=0x7fa45800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:52.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.253+0000 7fa45f7fe640 1 -- 172.21.15.53:0/823827149 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa458017070 con 0x7fa468106810 2024-09-19T06:16:52.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.253+0000 7fa4703a2640 1 -- 172.21.15.53:0/823827149 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa468199530 con 0x7fa468106810 2024-09-19T06:16:52.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.253+0000 7fa45f7fe640 1 -- 172.21.15.53:0/823827149 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa45800de00 con 0x7fa468106810 2024-09-19T06:16:52.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.253+0000 7fa45f7fe640 1 -- 172.21.15.53:0/823827149 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa458004aa0 con 0x7fa468106810 2024-09-19T06:16:52.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.253+0000 7fa4703a2640 1 -- 172.21.15.53:0/823827149 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4681a1b30 con 0x7fa468106810 2024-09-19T06:16:52.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.254+0000 7fa4703a2640 1 -- 172.21.15.53:0/823827149 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa468101f50 con 0x7fa468106810 2024-09-19T06:16:52.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.254+0000 7fa45f7fe640 1 -- 172.21.15.53:0/823827149 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7fa458004d50 con 0x7fa468106810 2024-09-19T06:16:52.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.255+0000 7fa45f7fe640 1 --2- 172.21.15.53:0/823827149 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fa45406e2b0 0x7fa454070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:52.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.255+0000 7fa45f7fe640 1 -- 172.21.15.53:0/823827149 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7224+0+0 (secure 0 0 0) 0x7fa45808e4f0 con 0x7fa468106810 2024-09-19T06:16:52.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.255+0000 7fa46e13e640 1 --2- 172.21.15.53:0/823827149 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fa45406e2b0 0x7fa454070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:52.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.255+0000 7fa46e13e640 1 --2- 172.21.15.53:0/823827149 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fa45406e2b0 0x7fa454070770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa464005ab0 tx=0x7fa464005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:52.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.259+0000 7fa45f7fe640 1 -- 172.21.15.53:0/823827149 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa45805b6e0 con 0x7fa468106810 2024-09-19T06:16:52.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.476+0000 7fa4703a2640 1 -- 172.21.15.53:0/823827149 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa468108d80 con 0x7fa468106810 2024-09-19T06:16:52.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.477+0000 7fa45f7fe640 1 -- 172.21.15.53:0/823827149 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+5765 (secure 0 0 0) 0x7fa45805f390 con 0x7fa468106810 2024-09-19T06:16:52.479 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:16:52.479 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":11,"btime":"2024-09-19T06:16:51:844508+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24345,"name":"foo.smithi155.dsbavw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/669003188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":669003188},{"type":"v1","addr":"172.21.15.155:6833","nonce":669003188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T06:16:32.878153+0000","modified":"2024-09-19T06:16:40.819179+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":14510},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14510":{"gid":14510,"name":"foo.smithi053.rdifzk","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/2573849324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2573849324},{"type":"v1","addr":"172.21.15.53:6835","nonce":2573849324}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"foo","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14510,"qdb_cluster":[14510]},"id":1},{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T06:16:36.000321+0000","modified":"2024-09-19T06:16:50.858313+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":14548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14548":{"gid":14548,"name":"bar.smithi053.nzdkgr","rank":0,"incarnation":10,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.53:6837/4281289831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":4281289831},{"type":"v1","addr":"172.21.15.53:6837","nonce":4281289831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"bar","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-09-19T06:16:52.480 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 11 2024-09-19T06:16:52.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.479+0000 7fa4703a2640 1 -- 172.21.15.53:0/823827149 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fa45406e2b0 msgr2=0x7fa454070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:52.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.479+0000 7fa4703a2640 1 --2- 172.21.15.53:0/823827149 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fa45406e2b0 0x7fa454070770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa464005ab0 tx=0x7fa464005a20 comp rx=0 tx=0).stop 2024-09-19T06:16:52.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.479+0000 7fa4703a2640 1 -- 172.21.15.53:0/823827149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa468106810 msgr2=0x7fa468198cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:52.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.479+0000 7fa4703a2640 1 --2- 172.21.15.53:0/823827149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa468106810 0x7fa468198cd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa45800d6f0 tx=0x7fa45800dbb0 comp rx=0 tx=0).stop 2024-09-19T06:16:52.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.479+0000 7fa4703a2640 1 -- 172.21.15.53:0/823827149 shutdown_connections 2024-09-19T06:16:52.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.479+0000 7fa4703a2640 1 --2- 172.21.15.53:0/823827149 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fa45406e2b0 0x7fa454070770 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:52.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.479+0000 7fa4703a2640 1 --2- 172.21.15.53:0/823827149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa468106810 0x7fa468198cd0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:52.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.479+0000 7fa4703a2640 1 --2- 172.21.15.53:0/823827149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa468100810 0x7fa468198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:52.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.479+0000 7fa4703a2640 1 -- 172.21.15.53:0/823827149 >> 172.21.15.53:0/823827149 conn(0x7fa4680fc510 msgr2=0x7fa4680fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:52.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.479+0000 7fa4703a2640 1 -- 172.21.15.53:0/823827149 shutdown_connections 2024-09-19T06:16:52.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:52.479+0000 7fa4703a2640 1 -- 172.21.15.53:0/823827149 wait complete. 2024-09-19T06:16:53.103 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 11, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T06:16:36.000321+0000', 'modified': '2024-09-19T06:16:50.858313+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 70, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 14548}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14548': {'gid': 14548, 'name': 'bar.smithi053.nzdkgr', 'rank': 0, 'incarnation': 10, 'state': 'up:reconnect', 'state_seq': 2, 'addr': '172.21.15.53:6837/4281289831', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6836', 'nonce': 4281289831}, {'type': 'v1', 'addr': '172.21.15.53:6837', 'nonce': 4281289831}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'bar', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-19T06:16:53.103 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_14548:up:reconnect 2024-09-19T06:16:53.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:53 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/823827149' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:16:53.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:53 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:53.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:53 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:53.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:53 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:53.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:53 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:53.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:53 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:53.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:53 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:16:53.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:53 smithi053 ceph-mon[27968]: mds.? [v2:172.21.15.53:6836/4281289831,v1:172.21.15.53:6837/4281289831] up:rejoin 2024-09-19T06:16:53.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:53 smithi053 ceph-mon[27968]: mds.? [v2:172.21.15.155:6834/3544520125,v1:172.21.15.155:6835/3544520125] up:boot 2024-09-19T06:16:53.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:53 smithi053 ceph-mon[27968]: fsmap foo:1 bar:1/1 {bar:0=bar.smithi053.nzdkgr=up:rejoin,foo:0=foo.smithi053.rdifzk=up:active} 2 up:standby 2024-09-19T06:16:53.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:53 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mds metadata", "who": "bar.smithi155.foldmc"}]: dispatch 2024-09-19T06:16:53.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:53 smithi053 ceph-mon[27968]: daemon mds.bar.smithi053.nzdkgr is now active in filesystem bar as rank 0 2024-09-19T06:16:53.385 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:53 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/823827149' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:16:53.385 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:53.385 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:53.385 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:53.385 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:53.385 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:53.385 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:16:53.385 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:53 smithi155 ceph-mon[35292]: mds.? [v2:172.21.15.53:6836/4281289831,v1:172.21.15.53:6837/4281289831] up:rejoin 2024-09-19T06:16:53.385 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:53 smithi155 ceph-mon[35292]: mds.? [v2:172.21.15.155:6834/3544520125,v1:172.21.15.155:6835/3544520125] up:boot 2024-09-19T06:16:53.385 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:53 smithi155 ceph-mon[35292]: fsmap foo:1 bar:1/1 {bar:0=bar.smithi053.nzdkgr=up:rejoin,foo:0=foo.smithi053.rdifzk=up:active} 2 up:standby 2024-09-19T06:16:53.386 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:53 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "mds metadata", "who": "bar.smithi155.foldmc"}]: dispatch 2024-09-19T06:16:53.386 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:53 smithi155 ceph-mon[35292]: daemon mds.bar.smithi053.nzdkgr is now active in filesystem bar as rank 0 2024-09-19T06:16:54.104 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:16:54.663 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:54.785 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:54 smithi053 ceph-mon[27968]: pgmap v220: 129 pgs: 129 active+clean; 581 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 818 B/s rd, 409 B/s wr, 3 op/s 2024-09-19T06:16:54.785 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:54 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:54.785 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:54 smithi053 ceph-mon[27968]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T06:16:54.785 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:54 smithi053 ceph-mon[27968]: Cluster is now healthy 2024-09-19T06:16:54.785 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:54 smithi053 ceph-mon[27968]: mds.? [v2:172.21.15.53:6836/4281289831,v1:172.21.15.53:6837/4281289831] up:active 2024-09-19T06:16:54.785 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:54 smithi053 ceph-mon[27968]: fsmap foo:1 bar:1 {bar:0=bar.smithi053.nzdkgr=up:active,foo:0=foo.smithi053.rdifzk=up:active} 2 up:standby 2024-09-19T06:16:54.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:54 smithi155 ceph-mon[35292]: pgmap v220: 129 pgs: 129 active+clean; 581 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 818 B/s rd, 409 B/s wr, 3 op/s 2024-09-19T06:16:54.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:54 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:54.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:54 smithi155 ceph-mon[35292]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T06:16:54.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:54 smithi155 ceph-mon[35292]: Cluster is now healthy 2024-09-19T06:16:54.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:54 smithi155 ceph-mon[35292]: mds.? [v2:172.21.15.53:6836/4281289831,v1:172.21.15.53:6837/4281289831] up:active 2024-09-19T06:16:54.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:54 smithi155 ceph-mon[35292]: fsmap foo:1 bar:1 {bar:0=bar.smithi053.nzdkgr=up:active,foo:0=foo.smithi053.rdifzk=up:active} 2 up:standby 2024-09-19T06:16:55.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.266+0000 7f039bc39640 1 -- 172.21.15.53:0/1194043028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03941033d0 msgr2=0x7f039410b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:55.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.266+0000 7f039bc39640 1 --2- 172.21.15.53:0/1194043028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03941033d0 0x7f039410b8c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0384009880 tx=0x7f038402a530 comp rx=0 tx=0).stop 2024-09-19T06:16:55.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.266+0000 7f039bc39640 1 -- 172.21.15.53:0/1194043028 shutdown_connections 2024-09-19T06:16:55.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.266+0000 7f039bc39640 1 --2- 172.21.15.53:0/1194043028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03941033d0 0x7f039410b8c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:55.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.266+0000 7f039bc39640 1 --2- 172.21.15.53:0/1194043028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0394102ab0 0x7f0394102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:55.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.266+0000 7f039bc39640 1 -- 172.21.15.53:0/1194043028 >> 172.21.15.53:0/1194043028 conn(0x7f03940fc550 msgr2=0x7f03940fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:55.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.267+0000 7f039bc39640 1 -- 172.21.15.53:0/1194043028 shutdown_connections 2024-09-19T06:16:55.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.267+0000 7f039bc39640 1 -- 172.21.15.53:0/1194043028 wait complete. 2024-09-19T06:16:55.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.267+0000 7f039bc39640 1 Processor -- start 2024-09-19T06:16:55.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.267+0000 7f039bc39640 1 -- start start 2024-09-19T06:16:55.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.268+0000 7f039bc39640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0394102ab0 0x7f039419d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:55.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.268+0000 7f039bc39640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03941033d0 0x7f039419db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:55.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.268+0000 7f039bc39640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f039410ed50 con 0x7f03941033d0 2024-09-19T06:16:55.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.268+0000 7f039bc39640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f039410ebd0 con 0x7f0394102ab0 2024-09-19T06:16:55.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.268+0000 7f03991d4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03941033d0 0x7f039419db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:55.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.268+0000 7f03999d5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0394102ab0 0x7f039419d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:55.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.268+0000 7f03991d4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03941033d0 0x7f039419db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54198/0 (socket says 172.21.15.53:54198) 2024-09-19T06:16:55.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.268+0000 7f03991d4640 1 -- 172.21.15.53:0/758150105 learned_addr learned my addr 172.21.15.53:0/758150105 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:55.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.268+0000 7f03991d4640 1 -- 172.21.15.53:0/758150105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0394102ab0 msgr2=0x7f039419d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:55.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.268+0000 7f03991d4640 1 --2- 172.21.15.53:0/758150105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0394102ab0 0x7f039419d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:55.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.268+0000 7f03991d4640 1 -- 172.21.15.53:0/758150105 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0394197710 con 0x7f03941033d0 2024-09-19T06:16:55.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.268+0000 7f03991d4640 1 --2- 172.21.15.53:0/758150105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03941033d0 0x7f039419db80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f038402aa50 tx=0x7f03840045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:55.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.269+0000 7f038affd640 1 -- 172.21.15.53:0/758150105 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f038403f070 con 0x7f03941033d0 2024-09-19T06:16:55.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.269+0000 7f039bc39640 1 -- 172.21.15.53:0/758150105 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03941979a0 con 0x7f03941033d0 2024-09-19T06:16:55.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.269+0000 7f038affd640 1 -- 172.21.15.53:0/758150105 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0384004950 con 0x7f03941033d0 2024-09-19T06:16:55.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.269+0000 7f038affd640 1 -- 172.21.15.53:0/758150105 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0384035030 con 0x7f03941033d0 2024-09-19T06:16:55.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.269+0000 7f039bc39640 1 -- 172.21.15.53:0/758150105 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0394197ed0 con 0x7f03941033d0 2024-09-19T06:16:55.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.270+0000 7f039bc39640 1 -- 172.21.15.53:0/758150105 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f039406a580 con 0x7f03941033d0 2024-09-19T06:16:55.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.270+0000 7f038affd640 1 -- 172.21.15.53:0/758150105 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f0384042070 con 0x7f03941033d0 2024-09-19T06:16:55.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.271+0000 7f038affd640 1 --2- 172.21.15.53:0/758150105 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f038006e2b0 0x7f0380070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:55.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.271+0000 7f038affd640 1 -- 172.21.15.53:0/758150105 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7224+0+0 (secure 0 0 0) 0x7f03840af180 con 0x7f03941033d0 2024-09-19T06:16:55.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.271+0000 7f03999d5640 1 --2- 172.21.15.53:0/758150105 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f038006e2b0 0x7f0380070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:55.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.271+0000 7f03999d5640 1 --2- 172.21.15.53:0/758150105 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f038006e2b0 0x7f0380070770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0390009bd0 tx=0x7f0390005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:55.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.275+0000 7f038affd640 1 -- 172.21.15.53:0/758150105 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0384004b00 con 0x7f03941033d0 2024-09-19T06:16:55.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.490+0000 7f039bc39640 1 -- 172.21.15.53:0/758150105 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0394105fc0 con 0x7f03941033d0 2024-09-19T06:16:55.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.491+0000 7f038affd640 1 -- 172.21.15.53:0/758150105 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6618 (secure 0 0 0) 0x7f0384082070 con 0x7f03941033d0 2024-09-19T06:16:55.493 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:16:55.493 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":13,"btime":"2024-09-19T06:16:53:847024+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24345,"name":"foo.smithi155.dsbavw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/669003188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":669003188},{"type":"v1","addr":"172.21.15.155:6833","nonce":669003188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24353,"name":"bar.smithi155.foldmc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3544520125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3544520125},{"type":"v1","addr":"172.21.15.155:6835","nonce":3544520125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T06:16:32.878153+0000","modified":"2024-09-19T06:16:40.819179+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":14510},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14510":{"gid":14510,"name":"foo.smithi053.rdifzk","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/2573849324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2573849324},{"type":"v1","addr":"172.21.15.53:6835","nonce":2573849324}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"foo","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14510,"qdb_cluster":[14510]},"id":1},{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T06:16:36.000321+0000","modified":"2024-09-19T06:16:53.847017+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":14548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14548":{"gid":14548,"name":"bar.smithi053.nzdkgr","rank":0,"incarnation":10,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6837/4281289831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":4281289831},{"type":"v1","addr":"172.21.15.53:6837","nonce":4281289831}]},"join_fscid":2,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"bar","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14548,"qdb_cluster":[14548]},"id":2}]} 2024-09-19T06:16:55.493 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 13 2024-09-19T06:16:55.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.493+0000 7f039bc39640 1 -- 172.21.15.53:0/758150105 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f038006e2b0 msgr2=0x7f0380070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:55.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.493+0000 7f039bc39640 1 --2- 172.21.15.53:0/758150105 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f038006e2b0 0x7f0380070770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0390009bd0 tx=0x7f0390005f90 comp rx=0 tx=0).stop 2024-09-19T06:16:55.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.493+0000 7f039bc39640 1 -- 172.21.15.53:0/758150105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03941033d0 msgr2=0x7f039419db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:55.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.493+0000 7f039bc39640 1 --2- 172.21.15.53:0/758150105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03941033d0 0x7f039419db80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f038402aa50 tx=0x7f03840045e0 comp rx=0 tx=0).stop 2024-09-19T06:16:55.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.493+0000 7f039bc39640 1 -- 172.21.15.53:0/758150105 shutdown_connections 2024-09-19T06:16:55.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.493+0000 7f039bc39640 1 --2- 172.21.15.53:0/758150105 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f038006e2b0 0x7f0380070770 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:55.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.493+0000 7f039bc39640 1 --2- 172.21.15.53:0/758150105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03941033d0 0x7f039419db80 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:55.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.493+0000 7f039bc39640 1 --2- 172.21.15.53:0/758150105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0394102ab0 0x7f039419d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:55.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.493+0000 7f039bc39640 1 -- 172.21.15.53:0/758150105 >> 172.21.15.53:0/758150105 conn(0x7f03940fc550 msgr2=0x7f0394103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:55.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.493+0000 7f039bc39640 1 -- 172.21.15.53:0/758150105 shutdown_connections 2024-09-19T06:16:55.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:55.493+0000 7f039bc39640 1 -- 172.21.15.53:0/758150105 wait complete. 2024-09-19T06:16:56.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:55 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:56.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:55 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:56.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:55 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/758150105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:16:56.169 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 13, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T06:16:36.000321+0000', 'modified': '2024-09-19T06:16:53.847017+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 70, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 14548}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14548': {'gid': 14548, 'name': 'bar.smithi053.nzdkgr', 'rank': 0, 'incarnation': 10, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.53:6837/4281289831', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6836', 'nonce': 4281289831}, {'type': 'v1', 'addr': '172.21.15.53:6837', 'nonce': 4281289831}]}, 'join_fscid': 2, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'bar', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 14548, 'qdb_cluster': [14548]} 2024-09-19T06:16:56.169 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T06:16:56.169 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph tell mds.bar.smithi053.nzdkgr status 2024-09-19T06:16:56.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:55 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:56.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:55 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:56.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:55 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/758150105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:16:56.572 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:57.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:56 smithi053 ceph-mon[27968]: pgmap v221: 129 pgs: 129 active+clean; 581 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1 op/s 2024-09-19T06:16:57.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:56 smithi155 ceph-mon[35292]: pgmap v221: 129 pgs: 129 active+clean; 581 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1 op/s 2024-09-19T06:16:57.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.624+0000 7fb4c232a640 1 -- 172.21.15.53:0/1394239149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4bc1068f0 msgr2=0x7fb4bc106cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:57.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.624+0000 7fb4c232a640 1 --2- 172.21.15.53:0/1394239149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4bc1068f0 0x7fb4bc106cd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac009880 tx=0x7fb4ac02a4a0 comp rx=0 tx=0).stop 2024-09-19T06:16:57.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.624+0000 7fb4c232a640 1 -- 172.21.15.53:0/1394239149 shutdown_connections 2024-09-19T06:16:57.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.624+0000 7fb4c232a640 1 --2- 172.21.15.53:0/1394239149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4bc100f50 0x7fb4bc1013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:57.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.624+0000 7fb4c232a640 1 --2- 172.21.15.53:0/1394239149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4bc1068f0 0x7fb4bc106cd0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:57.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.624+0000 7fb4c232a640 1 -- 172.21.15.53:0/1394239149 >> 172.21.15.53:0/1394239149 conn(0x7fb4bc0fbc90 msgr2=0x7fb4bc0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:57.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.624+0000 7fb4c232a640 1 -- 172.21.15.53:0/1394239149 shutdown_connections 2024-09-19T06:16:57.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.624+0000 7fb4c232a640 1 -- 172.21.15.53:0/1394239149 wait complete. 2024-09-19T06:16:57.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.624+0000 7fb4c232a640 1 Processor -- start 2024-09-19T06:16:57.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.624+0000 7fb4c232a640 1 -- start start 2024-09-19T06:16:57.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.624+0000 7fb4c232a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4bc100f50 0x7fb4bc19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:57.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.625+0000 7fb4c232a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4bc1068f0 0x7fb4bc19cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:57.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.625+0000 7fb4c232a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb4bc10dd30 con 0x7fb4bc100f50 2024-09-19T06:16:57.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.625+0000 7fb4c232a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb4bc10dbb0 con 0x7fb4bc1068f0 2024-09-19T06:16:57.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.625+0000 7fb4bbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4bc100f50 0x7fb4bc19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:57.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.625+0000 7fb4bbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4bc100f50 0x7fb4bc19c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54224/0 (socket says 172.21.15.53:54224) 2024-09-19T06:16:57.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.625+0000 7fb4bbfff640 1 -- 172.21.15.53:0/135278825 learned_addr learned my addr 172.21.15.53:0/135278825 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:57.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.625+0000 7fb4b3fff640 1 --2- 172.21.15.53:0/135278825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4bc1068f0 0x7fb4bc19cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:57.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.625+0000 7fb4bbfff640 1 -- 172.21.15.53:0/135278825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4bc1068f0 msgr2=0x7fb4bc19cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:57.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.625+0000 7fb4bbfff640 1 --2- 172.21.15.53:0/135278825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4bc1068f0 0x7fb4bc19cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:57.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.625+0000 7fb4bbfff640 1 -- 172.21.15.53:0/135278825 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4bc1966f0 con 0x7fb4bc100f50 2024-09-19T06:16:57.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.625+0000 7fb4b3fff640 1 --2- 172.21.15.53:0/135278825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4bc1068f0 0x7fb4bc19cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:16:57.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.625+0000 7fb4bbfff640 1 --2- 172.21.15.53:0/135278825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4bc100f50 0x7fb4bc19c620 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac006010 tx=0x7fb4ac0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:57.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.625+0000 7fb4b9ffb640 1 -- 172.21.15.53:0/135278825 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb4ac035070 con 0x7fb4bc100f50 2024-09-19T06:16:57.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.625+0000 7fb4b9ffb640 1 -- 172.21.15.53:0/135278825 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4ac004c20 con 0x7fb4bc100f50 2024-09-19T06:16:57.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.625+0000 7fb4b9ffb640 1 -- 172.21.15.53:0/135278825 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb4ac038830 con 0x7fb4bc100f50 2024-09-19T06:16:57.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.625+0000 7fb4c232a640 1 -- 172.21.15.53:0/135278825 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4bc196980 con 0x7fb4bc100f50 2024-09-19T06:16:57.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.625+0000 7fb4c232a640 1 -- 172.21.15.53:0/135278825 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4bc196eb0 con 0x7fb4bc100f50 2024-09-19T06:16:57.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.627+0000 7fb4b9ffb640 1 -- 172.21.15.53:0/135278825 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7fb4ac003bc0 con 0x7fb4bc100f50 2024-09-19T06:16:57.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.627+0000 7fb4b9ffb640 1 --2- 172.21.15.53:0/135278825 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fb4a806e310 0x7fb4a80707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:57.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.627+0000 7fb4b9ffb640 1 -- 172.21.15.53:0/135278825 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7224+0+0 (secure 0 0 0) 0x7fb4ac03e070 con 0x7fb4bc100f50 2024-09-19T06:16:57.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.627+0000 7fb4b3fff640 1 --2- 172.21.15.53:0/135278825 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fb4a806e310 0x7fb4a80707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:57.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.628+0000 7fb4c4703740 1 Processor -- start 2024-09-19T06:16:57.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.628+0000 7fb4b3fff640 1 --2- 172.21.15.53:0/135278825 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fb4a806e310 0x7fb4a80707d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc197f40 tx=0x7fb4a4005d30 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:57.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.628+0000 7fb4c4703740 1 -- start start 2024-09-19T06:16:57.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.628+0000 7fb4c4703740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9903b7310 0x55b9903b76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:57.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.628+0000 7fb4c4703740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9903b7c30 0x55b9903c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:57.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.628+0000 7fb4c4703740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b99031d180 con 0x55b9903b7310 2024-09-19T06:16:57.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.628+0000 7fb4c4703740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b9902c3d40 con 0x55b9903b7c30 2024-09-19T06:16:57.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.628+0000 7fb4bbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9903b7310 0x55b9903b76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:57.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.628+0000 7fb4bbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9903b7310 0x55b9903b76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54228/0 (socket says 172.21.15.53:54228) 2024-09-19T06:16:57.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.628+0000 7fb4bbfff640 1 -- 172.21.15.53:0/12124286 learned_addr learned my addr 172.21.15.53:0/12124286 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:57.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.628+0000 7fb4b3fff640 1 --2- 172.21.15.53:0/12124286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9903b7c30 0x55b9903c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:57.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.628+0000 7fb4bbfff640 1 -- 172.21.15.53:0/12124286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9903b7c30 msgr2=0x55b9903c64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:57.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.628+0000 7fb4bbfff640 1 --2- 172.21.15.53:0/12124286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9903b7c30 0x55b9903c64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:57.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.628+0000 7fb4bbfff640 1 -- 172.21.15.53:0/12124286 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b99008e420 con 0x55b9903b7310 2024-09-19T06:16:57.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.629+0000 7fb4bbfff640 1 --2- 172.21.15.53:0/12124286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9903b7310 0x55b9903b76f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac07d8e0 tx=0x7fb4ac02a4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a26e6fab7b9c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:57.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.629+0000 7fb4b27fc640 1 -- 172.21.15.53:0/12124286 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb4ac081d00 con 0x55b9903b7310 2024-09-19T06:16:57.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.629+0000 7fb4b27fc640 1 -- 172.21.15.53:0/12124286 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4ac086530 con 0x55b9903b7310 2024-09-19T06:16:57.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.629+0000 7fb4b27fc640 1 -- 172.21.15.53:0/12124286 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb4ac040370 con 0x55b9903b7310 2024-09-19T06:16:57.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.629+0000 7fb4c4703740 1 -- 172.21.15.53:0/12124286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9903b7310 msgr2=0x55b9903b76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:57.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.629+0000 7fb4c4703740 1 --2- 172.21.15.53:0/12124286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9903b7310 0x55b9903b76f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac07d8e0 tx=0x7fb4ac02a4a0 comp rx=0 tx=0).stop 2024-09-19T06:16:57.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.629+0000 7fb4c4703740 1 -- 172.21.15.53:0/12124286 shutdown_connections 2024-09-19T06:16:57.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.629+0000 7fb4c4703740 1 --2- 172.21.15.53:0/12124286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9903b7c30 0x55b9903c64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:57.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.629+0000 7fb4c4703740 1 --2- 172.21.15.53:0/12124286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9903b7310 0x55b9903b76f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:57.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.629+0000 7fb4c4703740 1 -- 172.21.15.53:0/12124286 >> 172.21.15.53:0/12124286 conn(0x55b9903b39e0 msgr2=0x55b9903b3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:57.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.630+0000 7fb4c4703740 1 -- 172.21.15.53:0/12124286 shutdown_connections 2024-09-19T06:16:57.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.630+0000 7fb4c4703740 1 -- 172.21.15.53:0/12124286 wait complete. 2024-09-19T06:16:57.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.630+0000 7fb4c4703740 1 -- start start 2024-09-19T06:16:57.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.630+0000 7fb4c4703740 1 Processor -- start 2024-09-19T06:16:57.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.630+0000 7fb4c4703740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9903b7310 0x55b9903c6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:57.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.630+0000 7fb4c4703740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9903b7c30 0x55b9903c72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:57.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.630+0000 7fb4c4703740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b9902b7ac0 con 0x55b9903b7c30 2024-09-19T06:16:57.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.630+0000 7fb4c4703740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b990319140 con 0x55b9903b7310 2024-09-19T06:16:57.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.630+0000 7fb4c4703740 10 client.0 _ll_register_callbacks cb 0x55b99002b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:57.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.630+0000 7fb4b3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9903b7c30 0x55b9903c72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:57.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.630+0000 7fb4b3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9903b7c30 0x55b9903c72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54242/0 (socket says 172.21.15.53:54242) 2024-09-19T06:16:57.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.630+0000 7fb4b3fff640 1 -- 172.21.15.53:0/539851885 learned_addr learned my addr 172.21.15.53:0/539851885 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:57.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.631+0000 7fb4bbfff640 1 --2- 172.21.15.53:0/539851885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9903b7310 0x55b9903c6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:57.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.631+0000 7fb4b3fff640 1 -- 172.21.15.53:0/539851885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9903b7310 msgr2=0x55b9903c6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:57.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.631+0000 7fb4b3fff640 1 --2- 172.21.15.53:0/539851885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9903b7310 0x55b9903c6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:57.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.631+0000 7fb4b3fff640 1 -- 172.21.15.53:0/539851885 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9903c7810 con 0x55b9903b7c30 2024-09-19T06:16:57.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.631+0000 7fb4b3fff640 1 --2- 172.21.15.53:0/539851885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9903b7c30 0x55b9903c72d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400f9a0 tx=0x7fb4a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac819e5e31849f4f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:57.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.631+0000 7fb49b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-19T06:16:57.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.631+0000 7fb49b7fe640 1 -- 172.21.15.53:0/539851885 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb4a400eac0 con 0x55b9903b7c30 2024-09-19T06:16:57.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.631+0000 7fb4c4703740 1 -- 172.21.15.53:0/539851885 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4ac086530 con 0x55b9903b7c30 2024-09-19T06:16:57.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.631+0000 7fb49b7fe640 1 -- 172.21.15.53:0/539851885 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 7224+0+0 (secure 0 0 0) 0x7fb4a400ec60 con 0x55b9903b7c30 2024-09-19T06:16:57.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.631+0000 7fb49b7fe640 1 -- 172.21.15.53:0/539851885 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7fb4a402c050 con 0x55b9903b7c30 2024-09-19T06:16:57.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.631+0000 7fb4c4703740 10 client.14568 fetch_fsmap learned FSMap version 14 2024-09-19T06:16:57.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.631+0000 7fb4c4703740 1 -- 172.21.15.53:0/539851885 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=3+}) -- 0x55b9903d20d0 con 0x55b9903b7c30 2024-09-19T06:16:57.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.631+0000 7fb49b7fe640 1 -- 172.21.15.53:0/539851885 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4ac086530 con 0x55b9903b7c30 2024-09-19T06:16:57.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.632+0000 7fb49b7fe640 1 -- 172.21.15.53:0/539851885 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 14) ==== 4216+0+0 (secure 0 0 0) 0x7fb4a402f020 con 0x55b9903b7c30 2024-09-19T06:16:57.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.632+0000 7fb4c4703740 10 client.14568 fetch_fsmap finished waiting for FSMap version 14 2024-09-19T06:16:57.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.632+0000 7fb4c4703740 10 client.14568 resolve_mds: resolved name 'bar.smithi053.nzdkgr' to daemon mds.bar.smithi053.nzdkgr 2024-09-19T06:16:57.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.632+0000 7fb4c4703740 20 client.14568 populate_metadata read hostname 'smithi053' 2024-09-19T06:16:57.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.632+0000 7fb4c4703740 1 --2- 172.21.15.53:0/539851885 >> [v2:172.21.15.53:6836/4281289831,v1:172.21.15.53:6837/4281289831] conn(0x55b9903d3140 0x55b9903d69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:57.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.632+0000 7fb4c4703740 4 client.14568 mds_command: new command op to 14548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:16:57.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.632+0000 7fb4c4703740 1 -- 172.21.15.53:0/539851885 --> [v2:172.21.15.53:6836/4281289831,v1:172.21.15.53:6837/4281289831] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b99032a6f0 con 0x55b9903d3140 2024-09-19T06:16:57.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.632+0000 7fb4bbfff640 1 --2- 172.21.15.53:0/539851885 >> [v2:172.21.15.53:6836/4281289831,v1:172.21.15.53:6837/4281289831] conn(0x55b9903d3140 0x55b9903d69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:57.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.632+0000 7fb4bbfff640 1 --2- 172.21.15.53:0/539851885 >> [v2:172.21.15.53:6836/4281289831,v1:172.21.15.53:6837/4281289831] conn(0x55b9903d3140 0x55b9903d69d0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a1cb23d9b83577d server_cookie=79e619c3fbb548f in_seq=0 out_seq=0 2024-09-19T06:16:57.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.632+0000 7fb49b7fe640 10 client.14568 ms_handle_connect on v2:172.21.15.53:6836/4281289831 2024-09-19T06:16:57.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.633+0000 7fb49b7fe640 1 -- 172.21.15.53:0/539851885 <== mds.0 v2:172.21.15.53:6836/4281289831 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b99032a6f0 con 0x55b9903d3140 2024-09-19T06:16:57.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.633+0000 7fb49b7fe640 10 client.14568 handle_command_reply: tid=0 2024-09-19T06:16:57.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.633+0000 7fb4c4703740 1 client.14568 shutdown 2024-09-19T06:16:57.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.633+0000 7fb4c4703740 1 -- 172.21.15.53:0/539851885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9903b7c30 msgr2=0x55b9903c72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:57.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.633+0000 7fb4c4703740 1 --2- 172.21.15.53:0/539851885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9903b7c30 0x55b9903c72d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400f9a0 tx=0x7fb4a40046c0 comp rx=0 tx=0).stop 2024-09-19T06:16:57.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.633+0000 7fb4c4703740 1 -- 172.21.15.53:0/539851885 shutdown_connections 2024-09-19T06:16:57.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.633+0000 7fb4c4703740 1 --2- 172.21.15.53:0/539851885 >> [v2:172.21.15.53:6836/4281289831,v1:172.21.15.53:6837/4281289831] conn(0x55b9903d3140 0x55b9903d69d0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:57.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.633+0000 7fb4c4703740 1 --2- 172.21.15.53:0/539851885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9903b7c30 0x55b9903c72d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:57.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.633+0000 7fb4c4703740 1 --2- 172.21.15.53:0/539851885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9903b7310 0x55b9903c6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:57.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.633+0000 7fb4c4703740 1 -- 172.21.15.53:0/539851885 >> 172.21.15.53:0/539851885 conn(0x55b9903b39e0 msgr2=0x55b9903b9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:57.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.633+0000 7fb49b7fe640 0 client.14568 ms_handle_reset on v2:172.21.15.53:6836/4281289831 2024-09-19T06:16:57.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.633+0000 7fb4c4703740 1 -- 172.21.15.53:0/539851885 shutdown_connections 2024-09-19T06:16:57.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.633+0000 7fb4c4703740 1 -- 172.21.15.53:0/539851885 wait complete. 2024-09-19T06:16:57.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.633+0000 7fb4c4703740 20 client.14568 trim_cache size 0 max 16384 2024-09-19T06:16:57.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.651+0000 7fb4c4703740 1 Processor -- start 2024-09-19T06:16:57.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.651+0000 7fb4c4703740 1 -- start start 2024-09-19T06:16:57.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.651+0000 7fb4c4703740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9903b7310 0x55b9903d9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:57.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.651+0000 7fb4c4703740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9903b7c30 0x55b9903d9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:57.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.651+0000 7fb4c4703740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb4bc1a8320 con 0x55b9903b7c30 2024-09-19T06:16:57.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.651+0000 7fb4c4703740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb4bc1a5cf0 con 0x55b9903b7310 2024-09-19T06:16:57.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.651+0000 7fb4b3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9903b7c30 0x55b9903d9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:57.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.651+0000 7fb4b3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9903b7c30 0x55b9903d9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54258/0 (socket says 172.21.15.53:54258) 2024-09-19T06:16:57.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.651+0000 7fb4b3fff640 1 -- 172.21.15.53:0/1508764301 learned_addr learned my addr 172.21.15.53:0/1508764301 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:57.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.652+0000 7fb4b3fff640 1 -- 172.21.15.53:0/1508764301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9903b7310 msgr2=0x55b9903d9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:57.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.652+0000 7fb4b3fff640 1 --2- 172.21.15.53:0/1508764301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9903b7310 0x55b9903d9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:57.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.652+0000 7fb4b3fff640 1 -- 172.21.15.53:0/1508764301 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9903b3810 con 0x55b9903b7c30 2024-09-19T06:16:57.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.652+0000 7fb4b3fff640 1 --2- 172.21.15.53:0/1508764301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9903b7c30 0x55b9903d9f40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400bfb0 tx=0x7fb4a400e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f25aff3ca11fa6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:57.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.652+0000 7fb4b1ffb640 1 -- 172.21.15.53:0/1508764301 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb4a400de10 con 0x55b9903b7c30 2024-09-19T06:16:57.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.652+0000 7fb4b1ffb640 1 -- 172.21.15.53:0/1508764301 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4a400eac0 con 0x55b9903b7c30 2024-09-19T06:16:57.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.652+0000 7fb4b1ffb640 1 -- 172.21.15.53:0/1508764301 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb4a4025b30 con 0x55b9903b7c30 2024-09-19T06:16:57.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.652+0000 7fb4c4703740 1 -- 172.21.15.53:0/1508764301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9903b7c30 msgr2=0x55b9903d9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:57.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.652+0000 7fb4c4703740 1 --2- 172.21.15.53:0/1508764301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9903b7c30 0x55b9903d9f40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400bfb0 tx=0x7fb4a400e830 comp rx=0 tx=0).stop 2024-09-19T06:16:57.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.653+0000 7fb4c4703740 1 -- 172.21.15.53:0/1508764301 shutdown_connections 2024-09-19T06:16:57.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.653+0000 7fb4c4703740 1 --2- 172.21.15.53:0/1508764301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9903b7c30 0x55b9903d9f40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:57.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.653+0000 7fb4c4703740 1 --2- 172.21.15.53:0/1508764301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9903b7310 0x55b9903d9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:57.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.653+0000 7fb4c4703740 1 -- 172.21.15.53:0/1508764301 >> 172.21.15.53:0/1508764301 conn(0x55b9903b39e0 msgr2=0x55b9903b9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:57.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.653+0000 7fb4c4703740 1 -- 172.21.15.53:0/1508764301 shutdown_connections 2024-09-19T06:16:57.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.653+0000 7fb4c4703740 1 -- 172.21.15.53:0/1508764301 wait complete. 2024-09-19T06:16:57.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.653+0000 7fb4c4703740 1 -- start start 2024-09-19T06:16:57.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.653+0000 7fb4c4703740 1 Processor -- start 2024-09-19T06:16:57.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.653+0000 7fb4c4703740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9903b7310 0x55b9904397a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:57.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.653+0000 7fb4c4703740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9903b7c30 0x55b990439ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:57.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.653+0000 7fb4c4703740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b9903d2fc0 con 0x55b9903b7310 2024-09-19T06:16:57.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.653+0000 7fb4c4703740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb4bc1a6340 con 0x55b9903b7c30 2024-09-19T06:16:57.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.653+0000 7fb4c4703740 10 client.0 _ll_register_callbacks cb 0x7fb4a4013070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:57.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.653+0000 7fb4bbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9903b7310 0x55b9904397a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:57.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.654+0000 7fb4bbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9903b7310 0x55b9904397a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54264/0 (socket says 172.21.15.53:54264) 2024-09-19T06:16:57.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.654+0000 7fb4bbfff640 1 -- 172.21.15.53:0/1088029305 learned_addr learned my addr 172.21.15.53:0/1088029305 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:57.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.654+0000 7fb4b3fff640 1 --2- 172.21.15.53:0/1088029305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9903b7c30 0x55b990439ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:57.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.654+0000 7fb4bbfff640 1 -- 172.21.15.53:0/1088029305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9903b7c30 msgr2=0x55b990439ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:57.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.654+0000 7fb4bbfff640 1 --2- 172.21.15.53:0/1088029305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9903b7c30 0x55b990439ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:57.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.654+0000 7fb4bbfff640 1 -- 172.21.15.53:0/1088029305 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9903d9550 con 0x55b9903b7310 2024-09-19T06:16:57.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.654+0000 7fb4bbfff640 1 --2- 172.21.15.53:0/1088029305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9903b7310 0x55b9904397a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac07d8e0 tx=0x7fb4ac079bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfa492bfdcc742f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:57.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.654+0000 7fb49b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-19T06:16:57.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.654+0000 7fb49b7fe640 1 -- 172.21.15.53:0/1088029305 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb4ac09c070 con 0x55b9903b7310 2024-09-19T06:16:57.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.654+0000 7fb49b7fe640 1 -- 172.21.15.53:0/1088029305 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 7224+0+0 (secure 0 0 0) 0x7fb4ac0840c0 con 0x55b9903b7310 2024-09-19T06:16:57.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.654+0000 7fb4c4703740 1 -- 172.21.15.53:0/1088029305 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4a400eac0 con 0x55b9903b7310 2024-09-19T06:16:57.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.654+0000 7fb49b7fe640 1 -- 172.21.15.53:0/1088029305 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7fb4ac040370 con 0x55b9903b7310 2024-09-19T06:16:57.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.654+0000 7fb4c4703740 10 client.14572 fetch_fsmap learned FSMap version 14 2024-09-19T06:16:57.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.654+0000 7fb4c4703740 1 -- 172.21.15.53:0/1088029305 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=3+}) -- 0x55b99043e5d0 con 0x55b9903b7310 2024-09-19T06:16:57.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.654+0000 7fb49b7fe640 1 -- 172.21.15.53:0/1088029305 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4a400eac0 con 0x55b9903b7310 2024-09-19T06:16:57.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.654+0000 7fb49b7fe640 1 -- 172.21.15.53:0/1088029305 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 14) ==== 4216+0+0 (secure 0 0 0) 0x7fb4ac0796a0 con 0x55b9903b7310 2024-09-19T06:16:57.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.654+0000 7fb4c4703740 10 client.14572 fetch_fsmap finished waiting for FSMap version 14 2024-09-19T06:16:57.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.655+0000 7fb4c4703740 10 client.14572 resolve_mds: resolved name 'bar.smithi053.nzdkgr' to daemon mds.bar.smithi053.nzdkgr 2024-09-19T06:16:57.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.655+0000 7fb4c4703740 20 client.14572 populate_metadata read hostname 'smithi053' 2024-09-19T06:16:57.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.655+0000 7fb4c4703740 1 --2- 172.21.15.53:0/1088029305 >> [v2:172.21.15.53:6836/4281289831,v1:172.21.15.53:6837/4281289831] conn(0x55b99043fd00 0x55b9904420f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:57.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.655+0000 7fb4c4703740 4 client.14572 mds_command: new command op to 14548 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:16:57.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.655+0000 7fb4c4703740 1 -- 172.21.15.53:0/1088029305 --> [v2:172.21.15.53:6836/4281289831,v1:172.21.15.53:6837/4281289831] -- command(tid 0: {"prefix": "status"}) -- 0x55b990324c00 con 0x55b99043fd00 2024-09-19T06:16:57.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.655+0000 7fb4b3fff640 1 --2- 172.21.15.53:0/1088029305 >> [v2:172.21.15.53:6836/4281289831,v1:172.21.15.53:6837/4281289831] conn(0x55b99043fd00 0x55b9904420f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:57.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.655+0000 7fb4b3fff640 1 --2- 172.21.15.53:0/1088029305 >> [v2:172.21.15.53:6836/4281289831,v1:172.21.15.53:6837/4281289831] conn(0x55b99043fd00 0x55b9904420f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43944cbdec247331 server_cookie=db771b90638c5ae0 in_seq=0 out_seq=0 2024-09-19T06:16:57.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.655+0000 7fb49b7fe640 10 client.14572 ms_handle_connect on v2:172.21.15.53:6836/4281289831 2024-09-19T06:16:57.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.655+0000 7fb49b7fe640 1 -- 172.21.15.53:0/1088029305 <== mds.0 v2:172.21.15.53:6836/4281289831 1 ==== command_reply(tid 0: 0 ) ==== 8+0+329 (crc 0 0 0) 0x55b990324c00 con 0x55b99043fd00 2024-09-19T06:16:57.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.655+0000 7fb49b7fe640 10 client.14572 handle_command_reply: tid=0 2024-09-19T06:16:57.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.655+0000 7fb4c4703740 1 client.14572 shutdown 2024-09-19T06:16:57.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.655+0000 7fb4c4703740 1 -- 172.21.15.53:0/1088029305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9903b7310 msgr2=0x55b9904397a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:57.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.655+0000 7fb4c4703740 1 --2- 172.21.15.53:0/1088029305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9903b7310 0x55b9904397a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac07d8e0 tx=0x7fb4ac079bf0 comp rx=0 tx=0).stop 2024-09-19T06:16:57.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.655+0000 7fb4c4703740 1 -- 172.21.15.53:0/1088029305 shutdown_connections 2024-09-19T06:16:57.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.655+0000 7fb4c4703740 1 --2- 172.21.15.53:0/1088029305 >> [v2:172.21.15.53:6836/4281289831,v1:172.21.15.53:6837/4281289831] conn(0x55b99043fd00 0x55b9904420f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:57.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.655+0000 7fb4c4703740 1 --2- 172.21.15.53:0/1088029305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9903b7c30 0x55b990439ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:57.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.655+0000 7fb4c4703740 1 --2- 172.21.15.53:0/1088029305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9903b7310 0x55b9904397a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:57.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.655+0000 7fb4c4703740 1 -- 172.21.15.53:0/1088029305 >> 172.21.15.53:0/1088029305 conn(0x55b9903b39e0 msgr2=0x55b9903b2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:57.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.656+0000 7fb49b7fe640 0 client.14572 ms_handle_reset on v2:172.21.15.53:6836/4281289831 2024-09-19T06:16:57.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.656+0000 7fb4c4703740 1 -- 172.21.15.53:0/1088029305 shutdown_connections 2024-09-19T06:16:57.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.656+0000 7fb4c4703740 1 -- 172.21.15.53:0/1088029305 wait complete. 2024-09-19T06:16:57.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.656+0000 7fb4c4703740 20 client.14572 trim_cache size 0 max 16384 2024-09-19T06:16:57.685 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-19T06:16:57.685 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "9f2876d0-764d-11ef-bceb-c7b262605968", 2024-09-19T06:16:57.685 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2024-09-19T06:16:57.685 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 14548, 2024-09-19T06:16:57.685 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2024-09-19T06:16:57.685 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2024-09-19T06:16:57.685 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "bar", 2024-09-19T06:16:57.685 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 6.8085410509999997, 2024-09-19T06:16:57.685 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 13, 2024-09-19T06:16:57.685 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 70, 2024-09-19T06:16:57.686 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 70, 2024-09-19T06:16:57.686 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 7.7305163759999997 2024-09-19T06:16:57.686 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-19T06:16:57.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.656+0000 7fb4c232a640 1 -- 172.21.15.53:0/135278825 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fb4a806e310 msgr2=0x7fb4a80707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:57.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.656+0000 7fb4c232a640 1 --2- 172.21.15.53:0/135278825 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fb4a806e310 0x7fb4a80707d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc197f40 tx=0x7fb4a4005d30 comp rx=0 tx=0).stop 2024-09-19T06:16:57.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.656+0000 7fb4c232a640 1 -- 172.21.15.53:0/135278825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4bc100f50 msgr2=0x7fb4bc19c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:57.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.656+0000 7fb4c232a640 1 --2- 172.21.15.53:0/135278825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4bc100f50 0x7fb4bc19c620 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac006010 tx=0x7fb4ac0048f0 comp rx=0 tx=0).stop 2024-09-19T06:16:57.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.656+0000 7fb4c232a640 1 -- 172.21.15.53:0/135278825 shutdown_connections 2024-09-19T06:16:57.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.656+0000 7fb4c232a640 1 --2- 172.21.15.53:0/135278825 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fb4a806e310 0x7fb4a80707d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:57.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.656+0000 7fb4c232a640 1 --2- 172.21.15.53:0/135278825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4bc1068f0 0x7fb4bc19cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:57.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.656+0000 7fb4c232a640 1 --2- 172.21.15.53:0/135278825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4bc100f50 0x7fb4bc19c620 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:57.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.656+0000 7fb4c232a640 1 -- 172.21.15.53:0/135278825 >> 172.21.15.53:0/135278825 conn(0x7fb4bc0fbc90 msgr2=0x7fb4bc0fc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:57.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.656+0000 7fb4c232a640 1 -- 172.21.15.53:0/135278825 shutdown_connections 2024-09-19T06:16:57.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:57.656+0000 7fb4c232a640 1 -- 172.21.15.53:0/135278825 wait complete. 2024-09-19T06:16:58.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:57 smithi053 ceph-mon[27968]: pgmap v222: 129 pgs: 129 active+clean; 581 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-19T06:16:58.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:57 smithi053 ceph-mon[27968]: mds.? [v2:172.21.15.155:6834/3544520125,v1:172.21.15.155:6835/3544520125] up:standby 2024-09-19T06:16:58.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:57 smithi053 ceph-mon[27968]: fsmap foo:1 bar:1 {bar:0=bar.smithi053.nzdkgr=up:active,foo:0=foo.smithi053.rdifzk=up:active} 2 up:standby 2024-09-19T06:16:58.233 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-19T06:16:58.264 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-19T06:16:58.264 INFO:tasks.ceph_fuse:config is {'client.0': {'cephfs_name': 'foo'}} 2024-09-19T06:16:58.264 INFO:tasks.ceph_fuse:client.0 config is {'cephfs_name': 'foo'} 2024-09-19T06:16:58.264 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T06:16:58.264 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T06:16:58.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:16:58.265 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-09-19T06:16:58.265 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:57 smithi155 ceph-mon[35292]: pgmap v222: 129 pgs: 129 active+clean; 581 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-19T06:16:58.265 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:57 smithi155 ceph-mon[35292]: mds.? [v2:172.21.15.155:6834/3544520125,v1:172.21.15.155:6835/3544520125] up:standby 2024-09-19T06:16:58.266 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:57 smithi155 ceph-mon[35292]: fsmap foo:1 bar:1 {bar:0=bar.smithi053.nzdkgr=up:active,foo:0=foo.smithi053.rdifzk=up:active} 2 up:standby 2024-09-19T06:16:58.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:16:58.283 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link delete ceph-brx 2024-09-19T06:16:58.353 INFO:teuthology.orchestra.run.smithi053.stderr:Cannot find device "ceph-brx" 2024-09-19T06:16:58.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T06:16:58.354 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-19T06:16:58.354 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-19T06:16:58.355 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-19T06:16:58.927 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:16:59.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.341+0000 7f79a1bd4640 1 -- 172.21.15.53:0/1669742123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f799c1033d0 msgr2=0x7f799c10b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:59.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.341+0000 7f79a1bd4640 1 --2- 172.21.15.53:0/1669742123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f799c1033d0 0x7f799c10b8c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7984009880 tx=0x7f798402a530 comp rx=0 tx=0).stop 2024-09-19T06:16:59.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.341+0000 7f79a1bd4640 1 -- 172.21.15.53:0/1669742123 shutdown_connections 2024-09-19T06:16:59.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.341+0000 7f79a1bd4640 1 --2- 172.21.15.53:0/1669742123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f799c1033d0 0x7f799c10b8c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:59.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.341+0000 7f79a1bd4640 1 --2- 172.21.15.53:0/1669742123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f799c102ab0 0x7f799c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:59.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.341+0000 7f79a1bd4640 1 -- 172.21.15.53:0/1669742123 >> 172.21.15.53:0/1669742123 conn(0x7f799c0fc550 msgr2=0x7f799c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:59.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.341+0000 7f79a1bd4640 1 -- 172.21.15.53:0/1669742123 shutdown_connections 2024-09-19T06:16:59.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.341+0000 7f79a1bd4640 1 -- 172.21.15.53:0/1669742123 wait complete. 2024-09-19T06:16:59.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.341+0000 7f79a1bd4640 1 Processor -- start 2024-09-19T06:16:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.341+0000 7f79a1bd4640 1 -- start start 2024-09-19T06:16:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.342+0000 7f79a1bd4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f799c102ab0 0x7f799c100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.342+0000 7f79a1bd4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f799c1033d0 0x7f799c100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.342+0000 7f79a1bd4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f799c10edb0 con 0x7f799c1033d0 2024-09-19T06:16:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.342+0000 7f79a1bd4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f799c10ec30 con 0x7f799c102ab0 2024-09-19T06:16:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.342+0000 7f799affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f799c1033d0 0x7f799c100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.342+0000 7f799affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f799c1033d0 0x7f799c100c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54272/0 (socket says 172.21.15.53:54272) 2024-09-19T06:16:59.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.342+0000 7f799affd640 1 -- 172.21.15.53:0/19680868 learned_addr learned my addr 172.21.15.53:0/19680868 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:16:59.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.342+0000 7f799b7fe640 1 --2- 172.21.15.53:0/19680868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f799c102ab0 0x7f799c100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:59.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.342+0000 7f799affd640 1 -- 172.21.15.53:0/19680868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f799c102ab0 msgr2=0x7f799c100710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:59.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.342+0000 7f799affd640 1 --2- 172.21.15.53:0/19680868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f799c102ab0 0x7f799c100710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:59.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.342+0000 7f799affd640 1 -- 172.21.15.53:0/19680868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799c101190 con 0x7f799c1033d0 2024-09-19T06:16:59.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.342+0000 7f799affd640 1 --2- 172.21.15.53:0/19680868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f799c1033d0 0x7f799c100c50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7984006010 tx=0x7f79840045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:59.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.342+0000 7f7998ff9640 1 -- 172.21.15.53:0/19680868 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f798403f070 con 0x7f799c1033d0 2024-09-19T06:16:59.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.343+0000 7f7998ff9640 1 -- 172.21.15.53:0/19680868 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79840049d0 con 0x7f799c1033d0 2024-09-19T06:16:59.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.343+0000 7f79a1bd4640 1 -- 172.21.15.53:0/19680868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f799c101420 con 0x7f799c1033d0 2024-09-19T06:16:59.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.343+0000 7f7998ff9640 1 -- 172.21.15.53:0/19680868 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7984035030 con 0x7f799c1033d0 2024-09-19T06:16:59.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.343+0000 7f79a1bd4640 1 -- 172.21.15.53:0/19680868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f799c1a3d80 con 0x7f799c1033d0 2024-09-19T06:16:59.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.344+0000 7f79a1bd4640 1 -- 172.21.15.53:0/19680868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f799c06a580 con 0x7f799c1033d0 2024-09-19T06:16:59.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.344+0000 7f7998ff9640 1 -- 172.21.15.53:0/19680868 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f7984042070 con 0x7f799c1033d0 2024-09-19T06:16:59.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.345+0000 7f7998ff9640 1 --2- 172.21.15.53:0/19680868 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f798806e310 0x7f79880707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:59.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.345+0000 7f7998ff9640 1 -- 172.21.15.53:0/19680868 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7224+0+0 (secure 0 0 0) 0x7f79840af2f0 con 0x7f799c1033d0 2024-09-19T06:16:59.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.345+0000 7f799b7fe640 1 --2- 172.21.15.53:0/19680868 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f798806e310 0x7f79880707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:59.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.345+0000 7f799b7fe640 1 --2- 172.21.15.53:0/19680868 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f798806e310 0x7f79880707d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f798c009f20 tx=0x7f798c0094f0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:59.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.349+0000 7f7998ff9640 1 -- 172.21.15.53:0/19680868 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f798407c560 con 0x7f799c1033d0 2024-09-19T06:16:59.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.612+0000 7f79a1bd4640 1 -- 172.21.15.53:0/19680868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f799c106030 con 0x7f799c1033d0 2024-09-19T06:16:59.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.613+0000 7f7998ff9640 1 -- 172.21.15.53:0/19680868 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) ==== 53+0+148 (secure 0 0 0) 0x7f7984080210 con 0x7f799c1033d0 2024-09-19T06:16:59.614 INFO:teuthology.orchestra.run.smithi053.stdout:name: foo, metadata pool: cephfs.foo.meta, data pools: [cephfs.foo.data ] 2024-09-19T06:16:59.615 INFO:teuthology.orchestra.run.smithi053.stdout:name: bar, metadata pool: cephfs.bar.meta, data pools: [cephfs.bar.data ] 2024-09-19T06:16:59.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.615+0000 7f79a1bd4640 1 -- 172.21.15.53:0/19680868 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f798806e310 msgr2=0x7f79880707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:59.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.615+0000 7f79a1bd4640 1 --2- 172.21.15.53:0/19680868 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f798806e310 0x7f79880707d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f798c009f20 tx=0x7f798c0094f0 comp rx=0 tx=0).stop 2024-09-19T06:16:59.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.615+0000 7f79a1bd4640 1 -- 172.21.15.53:0/19680868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f799c1033d0 msgr2=0x7f799c100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:59.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.615+0000 7f79a1bd4640 1 --2- 172.21.15.53:0/19680868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f799c1033d0 0x7f799c100c50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7984006010 tx=0x7f79840045e0 comp rx=0 tx=0).stop 2024-09-19T06:16:59.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.615+0000 7f79a1bd4640 1 -- 172.21.15.53:0/19680868 shutdown_connections 2024-09-19T06:16:59.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.615+0000 7f79a1bd4640 1 --2- 172.21.15.53:0/19680868 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f798806e310 0x7f79880707d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:59.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.615+0000 7f79a1bd4640 1 --2- 172.21.15.53:0/19680868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f799c1033d0 0x7f799c100c50 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:59.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.615+0000 7f79a1bd4640 1 --2- 172.21.15.53:0/19680868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f799c102ab0 0x7f799c100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:59.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.615+0000 7f79a1bd4640 1 -- 172.21.15.53:0/19680868 >> 172.21.15.53:0/19680868 conn(0x7f799c0fc550 msgr2=0x7f799c103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:59.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.615+0000 7f79a1bd4640 1 -- 172.21.15.53:0/19680868 shutdown_connections 2024-09-19T06:16:59.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:16:59.615+0000 7f79a1bd4640 1 -- 172.21.15.53:0/19680868 wait complete. 2024-09-19T06:16:59.994 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:59.995 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:16:59.995 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:59.995 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:59.995 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:16:59.995 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:16:59.995 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:16:59.995 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:59 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/19680868' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T06:16:59.995 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:16:59.995 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:16:59.995 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:16:59.995 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:16:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:17:00.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:17:00.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:17:00.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:17:00.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:17:00.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:17:00.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:17:00.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:17:00.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:59 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/19680868' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T06:17:00.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:17:00.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:17:00.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:17:00.234 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:16:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:17:00.294 INFO:tasks.cephfs.mount:Mounting Ceph FS foo; just confirmed its presence on cluster 2024-09-19T06:17:00.295 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T06:17:00.295 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2024-09-19T06:17:00.295 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T06:17:00.295 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T06:17:00.295 INFO:tasks.cephfs.mount:self.cephfs_name = foo 2024-09-19T06:17:00.295 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T06:17:00.295 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T06:17:00.295 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T06:17:00.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:17:00.295 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2024-09-19T06:17:00.313 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T06:17:00.313 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T06:17:00.313 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T06:17:00.313 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-09-19T06:17:00.313 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2024-09-19T06:17:00.313 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-09-19T06:17:00.313 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T06:17:00.313 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2024-09-19T06:17:00.313 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2024-09-19T06:17:00.314 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T06:17:00.314 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2024-09-19T06:17:00.314 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2024-09-19T06:17:00.314 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T06:17:00.314 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2024-09-19T06:17:00.314 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2024-09-19T06:17:00.314 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-19T06:17:00.314 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 42308sec preferred_lft 42308sec 2024-09-19T06:17:00.314 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link 2024-09-19T06:17:00.314 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-09-19T06:17:00.314 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T06:17:00.314 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2024-09-19T06:17:00.314 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2024-09-19T06:17:00.315 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-19T06:17:00.315 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T06:17:00.315 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-09-19T06:17:00.315 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link add name ceph-brx type bridge 2024-09-19T06:17:00.315 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip addr flush dev ceph-brx 2024-09-19T06:17:00.316 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set ceph-brx up 2024-09-19T06:17:00.316 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-19T06:17:00.316 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-09-19T06:17:00.406 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T06:17:00.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:17:00.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:17:00.530 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T06:17:00.557 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-09-19T06:17:00.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:17:00.559 DEBUG:teuthology.orchestra.run.smithi053:> route 2024-09-19T06:17:00.629 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2024-09-19T06:17:00.629 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T06:17:00.629 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-19T06:17:00.629 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-19T06:17:00.629 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T06:17:00.630 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T06:17:00.630 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-09-19T06:17:00.630 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-19T06:17:00.630 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-19T06:17:00.630 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-19T06:17:00.630 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-09-19T06:17:00.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T06:17:00.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:17:00.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:17:00.951 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-09-19T06:17:00.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:17:00.968 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list-id 2024-09-19T06:17:01.023 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T06:17:01.023 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-09-19T06:17:01.023 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T06:17:01.024 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-19T06:17:01.024 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-09-19T06:17:01.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:00 smithi053 ceph-mon[27968]: pgmap v223: 129 pgs: 129 active+clean; 581 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 8.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-19T06:17:01.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T06:17:01.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:17:01.143 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-19T06:17:01.144 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T06:17:01.144 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-09-19T06:17:01.144 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-19T06:17:01.144 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-19T06:17:01.144 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-19T06:17:01.144 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-19T06:17:01.144 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-19T06:17:01.144 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-09-19T06:17:01.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:00 smithi155 ceph-mon[35292]: pgmap v223: 129 pgs: 129 active+clean; 581 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 8.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-19T06:17:01.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T06:17:01.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:17:01.314 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T06:17:01.314 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-09-19T06:17:01.314 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2024-09-19T06:17:01.314 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set dev brx.0 master ceph-brx 2024-09-19T06:17:01.314 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-09-19T06:17:01.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T06:17:01.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:17:01.393 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'cephfs_name': 'foo'} 2024-09-19T06:17:01.393 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T06:17:01.394 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T06:17:01.409 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T06:17:01.410 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T06:17:01.410 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T06:17:01.465 DEBUG:teuthology.orchestra.run.smithi053:> sudo modprobe fuse 2024-09-19T06:17:01.532 DEBUG:teuthology.orchestra.run.smithi053:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T06:17:01.589 INFO:teuthology.orchestra.run.smithi053.stdout:/proc 2024-09-19T06:17:01.589 INFO:teuthology.orchestra.run.smithi053.stdout:/sys 2024-09-19T06:17:01.590 INFO:teuthology.orchestra.run.smithi053.stdout:/dev 2024-09-19T06:17:01.590 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/security 2024-09-19T06:17:01.590 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/shm 2024-09-19T06:17:01.590 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/pts 2024-09-19T06:17:01.590 INFO:teuthology.orchestra.run.smithi053.stdout:/run 2024-09-19T06:17:01.590 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/cgroup 2024-09-19T06:17:01.590 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/pstore 2024-09-19T06:17:01.590 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/bpf 2024-09-19T06:17:01.590 INFO:teuthology.orchestra.run.smithi053.stdout:/ 2024-09-19T06:17:01.590 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/selinux 2024-09-19T06:17:01.590 INFO:teuthology.orchestra.run.smithi053.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T06:17:01.590 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/debug 2024-09-19T06:17:01.590 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/mqueue 2024-09-19T06:17:01.591 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/hugepages 2024-09-19T06:17:01.591 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/tracing 2024-09-19T06:17:01.591 INFO:teuthology.orchestra.run.smithi053.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T06:17:01.591 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/fuse/connections 2024-09-19T06:17:01.591 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/config 2024-09-19T06:17:01.591 INFO:teuthology.orchestra.run.smithi053.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T06:17:01.591 INFO:teuthology.orchestra.run.smithi053.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T06:17:01.591 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T06:17:01.591 INFO:teuthology.orchestra.run.smithi053.stdout:/run/user/1000 2024-09-19T06:17:01.591 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/ceph 2024-09-19T06:17:01.591 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay 2024-09-19T06:17:01.591 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/d2f9f09b1f48a651ccdd968737ef3a081e1766c8413dec44faa7542d15e6e4ec/merged 2024-09-19T06:17:01.591 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/fdcc97a79881c7e5268b66a1628ca15dd0de697dae72455cc4a72b00338d30d5/merged 2024-09-19T06:17:01.591 INFO:teuthology.orchestra.run.smithi053.stdout:/run/user/0 2024-09-19T06:17:01.592 INFO:teuthology.orchestra.run.smithi053.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T06:17:01.592 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/74c1fecbba935daf61b8fc346f42ea5fa869cec3eb263512c4e89bf02b607e56/merged 2024-09-19T06:17:01.592 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/ae7613604d9abd182defb6d88563f1851a037934a3f97529b334b45bb53c7c3c/merged 2024-09-19T06:17:01.592 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/a816658b4a86d51683dc3724f9e5fc32de7456b6c93869a6ee7d9e789ba8224b/merged 2024-09-19T06:17:01.592 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/5a52dededbec74be706daf7110a780fb3cd39ea3b4ef7f266ca828dd43f98bcf/merged 2024-09-19T06:17:01.592 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/a121757f5050c2845314fcb409ace16b64531733bc578520f5ca788fd7e32635/merged 2024-09-19T06:17:01.592 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/ea991316be66cf2df0b42db33f3cb5faae616f6f70accdf06e5caa03547fd221/merged 2024-09-19T06:17:01.592 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/e5ef75b955272730be5fc2ddd5e539cc3ea309e8910ca0fd6dffae8b9e783016/merged 2024-09-19T06:17:01.592 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/c547cd5231027c601197986ecf6254ff69a15b572df09f9cf80bc870101fe024/merged 2024-09-19T06:17:01.592 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/e41c9b603e693ec4f9f2141cd12ffb8516aa4bcf28f584457ce7dd1596aaa69c/merged 2024-09-19T06:17:01.592 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/32a10f17d5ede08bf0ac8706e5874294f65c93d601490778b893cdfbbb501d96/merged 2024-09-19T06:17:01.592 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/3a08cd75f0291bcbfbbb041f18eb2ce1fee7e6ee52e2c401dfa92e4fe0c6e14e/merged 2024-09-19T06:17:01.592 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/f29ea1ea7e7e57fc673cfd38b79254ec0fce094be194ace9fe778619bf886b1f/merged 2024-09-19T06:17:01.593 INFO:teuthology.orchestra.run.smithi053.stdout:/run/netns 2024-09-19T06:17:01.593 INFO:teuthology.orchestra.run.smithi053.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T06:17:01.593 INFO:teuthology.orchestra.run.smithi053.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T06:17:01.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:17:01.593 DEBUG:teuthology.orchestra.run.smithi053:> ls /sys/fs/fuse/connections 2024-09-19T06:17:01.645 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T06:17:01.646 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_fs=foo) 2024-09-19T06:17:01.688 DEBUG:teuthology.orchestra.run.smithi053:> sudo modprobe fuse 2024-09-19T06:17:01.713 DEBUG:teuthology.orchestra.run.smithi053:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T06:17:01.770 INFO:teuthology.orchestra.run.smithi053.stdout:/proc 2024-09-19T06:17:01.770 INFO:teuthology.orchestra.run.smithi053.stdout:/sys 2024-09-19T06:17:01.771 INFO:teuthology.orchestra.run.smithi053.stdout:/dev 2024-09-19T06:17:01.771 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/security 2024-09-19T06:17:01.771 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/shm 2024-09-19T06:17:01.771 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/pts 2024-09-19T06:17:01.771 INFO:teuthology.orchestra.run.smithi053.stdout:/run 2024-09-19T06:17:01.771 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/cgroup 2024-09-19T06:17:01.771 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/pstore 2024-09-19T06:17:01.771 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/bpf 2024-09-19T06:17:01.771 INFO:teuthology.orchestra.run.smithi053.stdout:/ 2024-09-19T06:17:01.771 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/selinux 2024-09-19T06:17:01.771 INFO:teuthology.orchestra.run.smithi053.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T06:17:01.771 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/debug 2024-09-19T06:17:01.771 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/mqueue 2024-09-19T06:17:01.772 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/hugepages 2024-09-19T06:17:01.772 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/tracing 2024-09-19T06:17:01.772 INFO:teuthology.orchestra.run.smithi053.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T06:17:01.772 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/fuse/connections 2024-09-19T06:17:01.772 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/config 2024-09-19T06:17:01.772 INFO:teuthology.orchestra.run.smithi053.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T06:17:01.772 INFO:teuthology.orchestra.run.smithi053.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T06:17:01.772 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T06:17:01.772 INFO:teuthology.orchestra.run.smithi053.stdout:/run/user/1000 2024-09-19T06:17:01.772 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/ceph 2024-09-19T06:17:01.772 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay 2024-09-19T06:17:01.772 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/d2f9f09b1f48a651ccdd968737ef3a081e1766c8413dec44faa7542d15e6e4ec/merged 2024-09-19T06:17:01.772 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/fdcc97a79881c7e5268b66a1628ca15dd0de697dae72455cc4a72b00338d30d5/merged 2024-09-19T06:17:01.772 INFO:teuthology.orchestra.run.smithi053.stdout:/run/user/0 2024-09-19T06:17:01.773 INFO:teuthology.orchestra.run.smithi053.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T06:17:01.773 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/74c1fecbba935daf61b8fc346f42ea5fa869cec3eb263512c4e89bf02b607e56/merged 2024-09-19T06:17:01.773 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/ae7613604d9abd182defb6d88563f1851a037934a3f97529b334b45bb53c7c3c/merged 2024-09-19T06:17:01.773 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/a816658b4a86d51683dc3724f9e5fc32de7456b6c93869a6ee7d9e789ba8224b/merged 2024-09-19T06:17:01.773 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/5a52dededbec74be706daf7110a780fb3cd39ea3b4ef7f266ca828dd43f98bcf/merged 2024-09-19T06:17:01.773 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/a121757f5050c2845314fcb409ace16b64531733bc578520f5ca788fd7e32635/merged 2024-09-19T06:17:01.773 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/ea991316be66cf2df0b42db33f3cb5faae616f6f70accdf06e5caa03547fd221/merged 2024-09-19T06:17:01.773 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/e5ef75b955272730be5fc2ddd5e539cc3ea309e8910ca0fd6dffae8b9e783016/merged 2024-09-19T06:17:01.773 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/c547cd5231027c601197986ecf6254ff69a15b572df09f9cf80bc870101fe024/merged 2024-09-19T06:17:01.773 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/e41c9b603e693ec4f9f2141cd12ffb8516aa4bcf28f584457ce7dd1596aaa69c/merged 2024-09-19T06:17:01.773 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/32a10f17d5ede08bf0ac8706e5874294f65c93d601490778b893cdfbbb501d96/merged 2024-09-19T06:17:01.773 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/3a08cd75f0291bcbfbbb041f18eb2ce1fee7e6ee52e2c401dfa92e4fe0c6e14e/merged 2024-09-19T06:17:01.773 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/f29ea1ea7e7e57fc673cfd38b79254ec0fce094be194ace9fe778619bf886b1f/merged 2024-09-19T06:17:01.774 INFO:teuthology.orchestra.run.smithi053.stdout:/run/netns 2024-09-19T06:17:01.774 INFO:teuthology.orchestra.run.smithi053.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T06:17:01.774 INFO:teuthology.orchestra.run.smithi053.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T06:17:01.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:17:01.774 DEBUG:teuthology.orchestra.run.smithi053:> ls /sys/fs/fuse/connections 2024-09-19T06:17:01.821 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi053.stderr:2024-09-19T06:17:01.819+0000 7f4aa9d1e480 -1 init, newargv = 0x562254039060 newargc=15 2024-09-19T06:17:01.821 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi053.stderr:ceph-fuse[78081]: starting ceph client 2024-09-19T06:17:01.826 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi053.stderr:ceph-fuse[78081]: starting fuse 2024-09-19T06:17:01.827 INFO:teuthology.orchestra.run.smithi053.stdout:144 2024-09-19T06:17:01.827 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [144] 2024-09-19T06:17:01.828 DEBUG:teuthology.orchestra.run.smithi053:> sudo stdin-killer -- python3 -c ' 2024-09-19T06:17:01.828 DEBUG:teuthology.orchestra.run.smithi053:> import glob 2024-09-19T06:17:01.828 DEBUG:teuthology.orchestra.run.smithi053:> import re 2024-09-19T06:17:01.828 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-09-19T06:17:01.828 DEBUG:teuthology.orchestra.run.smithi053:> import subprocess 2024-09-19T06:17:01.828 DEBUG:teuthology.orchestra.run.smithi053:> 2024-09-19T06:17:01.828 DEBUG:teuthology.orchestra.run.smithi053:> def _find_admin_socket(client_name): 2024-09-19T06:17:01.828 DEBUG:teuthology.orchestra.run.smithi053:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T06:17:01.828 DEBUG:teuthology.orchestra.run.smithi053:> files = glob.glob(asok_path) 2024-09-19T06:17:01.828 DEBUG:teuthology.orchestra.run.smithi053:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T06:17:01.828 DEBUG:teuthology.orchestra.run.smithi053:> 2024-09-19T06:17:01.828 DEBUG:teuthology.orchestra.run.smithi053:> # Given a non-glob path, it better be there 2024-09-19T06:17:01.828 DEBUG:teuthology.orchestra.run.smithi053:> if "*" not in asok_path: 2024-09-19T06:17:01.829 DEBUG:teuthology.orchestra.run.smithi053:> assert(len(files) == 1) 2024-09-19T06:17:01.829 DEBUG:teuthology.orchestra.run.smithi053:> return files[0] 2024-09-19T06:17:01.829 DEBUG:teuthology.orchestra.run.smithi053:> 2024-09-19T06:17:01.829 DEBUG:teuthology.orchestra.run.smithi053:> for f in files: 2024-09-19T06:17:01.829 DEBUG:teuthology.orchestra.run.smithi053:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T06:17:01.829 DEBUG:teuthology.orchestra.run.smithi053:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T06:17:01.829 DEBUG:teuthology.orchestra.run.smithi053:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T06:17:01.829 DEBUG:teuthology.orchestra.run.smithi053:> contents = proc_f.read() 2024-09-19T06:17:01.829 DEBUG:teuthology.orchestra.run.smithi053:> if mountpoint in contents: 2024-09-19T06:17:01.829 DEBUG:teuthology.orchestra.run.smithi053:> return f 2024-09-19T06:17:01.829 DEBUG:teuthology.orchestra.run.smithi053:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T06:17:01.829 DEBUG:teuthology.orchestra.run.smithi053:> 2024-09-19T06:17:01.829 DEBUG:teuthology.orchestra.run.smithi053:> print(_find_admin_socket("client.0")) 2024-09-19T06:17:01.829 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-09-19T06:17:01.948 INFO:teuthology.orchestra.run.smithi053.stdout:/var/run/ceph/ceph-client.0.78081.asok 2024-09-19T06:17:01.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:17:01.959 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.78081.asok 2024-09-19T06:17:01.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:17:01.959 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.78081.asok status 2024-09-19T06:17:02.096 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-19T06:17:02.097 INFO:teuthology.orchestra.run.smithi053.stdout: "metadata": { 2024-09-19T06:17:02.097 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T06:17:02.097 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T06:17:02.097 INFO:teuthology.orchestra.run.smithi053.stdout: "entity_id": "0", 2024-09-19T06:17:02.097 INFO:teuthology.orchestra.run.smithi053.stdout: "hostname": "smithi053", 2024-09-19T06:17:02.097 INFO:teuthology.orchestra.run.smithi053.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T06:17:02.097 INFO:teuthology.orchestra.run.smithi053.stdout: "pid": "78081", 2024-09-19T06:17:02.097 INFO:teuthology.orchestra.run.smithi053.stdout: "root": "/" 2024-09-19T06:17:02.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-19T06:17:02.097 INFO:teuthology.orchestra.run.smithi053.stdout: "dentry_count": 0, 2024-09-19T06:17:02.097 INFO:teuthology.orchestra.run.smithi053.stdout: "dentry_pinned_count": 0, 2024-09-19T06:17:02.097 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 14580, 2024-09-19T06:17:02.097 INFO:teuthology.orchestra.run.smithi053.stdout: "inst": { 2024-09-19T06:17:02.098 INFO:teuthology.orchestra.run.smithi053.stdout: "name": { 2024-09-19T06:17:02.098 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "client", 2024-09-19T06:17:02.098 INFO:teuthology.orchestra.run.smithi053.stdout: "num": 14580 2024-09-19T06:17:02.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-19T06:17:02.098 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": { 2024-09-19T06:17:02.098 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-19T06:17:02.098 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "192.168.0.1:0", 2024-09-19T06:17:02.098 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 713266004 2024-09-19T06:17:02.098 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-19T06:17:02.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-19T06:17:02.098 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": { 2024-09-19T06:17:02.098 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-19T06:17:02.099 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "192.168.0.1:0", 2024-09-19T06:17:02.099 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 713266004 2024-09-19T06:17:02.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-19T06:17:02.099 INFO:teuthology.orchestra.run.smithi053.stdout: "inst_str": "client.14580 192.168.0.1:0/713266004", 2024-09-19T06:17:02.099 INFO:teuthology.orchestra.run.smithi053.stdout: "addr_str": "192.168.0.1:0/713266004", 2024-09-19T06:17:02.099 INFO:teuthology.orchestra.run.smithi053.stdout: "inode_count": 1, 2024-09-19T06:17:02.099 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_epoch": 8, 2024-09-19T06:17:02.099 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_epoch": 70, 2024-09-19T06:17:02.099 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_epoch_barrier": 0, 2024-09-19T06:17:02.099 INFO:teuthology.orchestra.run.smithi053.stdout: "blocklisted": false, 2024-09-19T06:17:02.100 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "foo" 2024-09-19T06:17:02.100 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-19T06:17:02.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:17:02.105 DEBUG:teuthology.orchestra.run.smithi053:> stat --file-system '--printf=%T 2024-09-19T06:17:02.105 DEBUG:teuthology.orchestra.run.smithi053:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T06:17:02.121 INFO:teuthology.orchestra.run.smithi053.stdout:fuseblk 2024-09-19T06:17:02.122 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T06:17:02.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:17:02.122 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T06:17:02.188 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-19T06:17:02.196 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-19T06:17:02.197 INFO:tasks.ceph_fuse:config is {'client.1': {'cephfs_name': 'bar'}} 2024-09-19T06:17:02.197 INFO:tasks.ceph_fuse:client.1 config is {'cephfs_name': 'bar'} 2024-09-19T06:17:02.197 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T06:17:02.197 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-19T06:17:02.198 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-19T06:17:02.198 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T06:17:02.247 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-19T06:17:02.248 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T06:17:02.306 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-19T06:17:02.608 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:17:02.987 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:02 smithi053 ceph-mon[27968]: pgmap v224: 129 pgs: 129 active+clean; 581 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 8.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-19T06:17:03.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.070+0000 7f759d2eb640 1 -- 172.21.15.53:0/1533923125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75980ffdc0 msgr2=0x7f7598100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:03.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.070+0000 7f759d2eb640 1 --2- 172.21.15.53:0/1533923125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75980ffdc0 0x7f7598100220 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f758c009880 tx=0x7f758c02a4a0 comp rx=0 tx=0).stop 2024-09-19T06:17:03.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.070+0000 7f759d2eb640 1 -- 172.21.15.53:0/1533923125 shutdown_connections 2024-09-19T06:17:03.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.070+0000 7f759d2eb640 1 --2- 172.21.15.53:0/1533923125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75980ffdc0 0x7f7598100220 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:03.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.070+0000 7f759d2eb640 1 --2- 172.21.15.53:0/1533923125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75981069a0 0x7f7598106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:03.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.070+0000 7f759d2eb640 1 -- 172.21.15.53:0/1533923125 >> 172.21.15.53:0/1533923125 conn(0x7f75980fbc90 msgr2=0x7f75980fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:03.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.070+0000 7f759d2eb640 1 -- 172.21.15.53:0/1533923125 shutdown_connections 2024-09-19T06:17:03.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.070+0000 7f759d2eb640 1 -- 172.21.15.53:0/1533923125 wait complete. 2024-09-19T06:17:03.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.071+0000 7f759d2eb640 1 Processor -- start 2024-09-19T06:17:03.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.071+0000 7f759d2eb640 1 -- start start 2024-09-19T06:17:03.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.071+0000 7f759d2eb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75980ffdc0 0x7f759819d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:03.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.071+0000 7f759d2eb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75981069a0 0x7f759819dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:03.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.071+0000 7f759d2eb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f759810ef10 con 0x7f75980ffdc0 2024-09-19T06:17:03.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.071+0000 7f759d2eb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f759810ed90 con 0x7f75981069a0 2024-09-19T06:17:03.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.072+0000 7f7596ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75980ffdc0 0x7f759819d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:03.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.072+0000 7f7596ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75980ffdc0 0x7f759819d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50572/0 (socket says 172.21.15.53:50572) 2024-09-19T06:17:03.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.072+0000 7f7596ffd640 1 -- 172.21.15.53:0/3289910606 learned_addr learned my addr 172.21.15.53:0/3289910606 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:17:03.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.072+0000 7f75967fc640 1 --2- 172.21.15.53:0/3289910606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75981069a0 0x7f759819dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:03.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.072+0000 7f7596ffd640 1 -- 172.21.15.53:0/3289910606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75981069a0 msgr2=0x7f759819dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:03.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.072+0000 7f7596ffd640 1 --2- 172.21.15.53:0/3289910606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75981069a0 0x7f759819dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:03.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.072+0000 7f7596ffd640 1 -- 172.21.15.53:0/3289910606 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759819e2e0 con 0x7f75980ffdc0 2024-09-19T06:17:03.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.072+0000 7f7596ffd640 1 --2- 172.21.15.53:0/3289910606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75980ffdc0 0x7f759819d6f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f758800b420 tx=0x7f758800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:03.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.072+0000 7f757ffff640 1 -- 172.21.15.53:0/3289910606 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7588011630 con 0x7f75980ffdc0 2024-09-19T06:17:03.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.072+0000 7f757ffff640 1 -- 172.21.15.53:0/3289910606 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7588016070 con 0x7f75980ffdc0 2024-09-19T06:17:03.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.072+0000 7f759d2eb640 1 -- 172.21.15.53:0/3289910606 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75981979d0 con 0x7f75980ffdc0 2024-09-19T06:17:03.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.072+0000 7f757ffff640 1 -- 172.21.15.53:0/3289910606 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7588010c90 con 0x7f75980ffdc0 2024-09-19T06:17:03.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.072+0000 7f759d2eb640 1 -- 172.21.15.53:0/3289910606 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7598197f30 con 0x7f75980ffdc0 2024-09-19T06:17:03.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.073+0000 7f759d2eb640 1 -- 172.21.15.53:0/3289910606 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75981030e0 con 0x7f75980ffdc0 2024-09-19T06:17:03.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.074+0000 7f757ffff640 1 -- 172.21.15.53:0/3289910606 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f75880117d0 con 0x7f75980ffdc0 2024-09-19T06:17:03.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.074+0000 7f757ffff640 1 --2- 172.21.15.53:0/3289910606 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f758006e300 0x7f75800707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:03.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.074+0000 7f757ffff640 1 -- 172.21.15.53:0/3289910606 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7224+0+0 (secure 0 0 0) 0x7f758808e0e0 con 0x7f75980ffdc0 2024-09-19T06:17:03.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.074+0000 7f75967fc640 1 --2- 172.21.15.53:0/3289910606 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f758006e300 0x7f75800707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:03.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.075+0000 7f75967fc640 1 --2- 172.21.15.53:0/3289910606 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f758006e300 0x7f75800707c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f75981990a0 tx=0x7f758c005d50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:03.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.079+0000 7f757ffff640 1 -- 172.21.15.53:0/3289910606 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f758805b350 con 0x7f75980ffdc0 2024-09-19T06:17:03.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:02 smithi155 ceph-mon[35292]: pgmap v224: 129 pgs: 129 active+clean; 581 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 8.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-19T06:17:03.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.264+0000 7f759d2eb640 1 -- 172.21.15.53:0/3289910606 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7598100220 con 0x7f75980ffdc0 2024-09-19T06:17:03.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.264+0000 7f757ffff640 1 -- 172.21.15.53:0/3289910606 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v70) ==== 81+0+1001 (secure 0 0 0) 0x7f758805f000 con 0x7f75980ffdc0 2024-09-19T06:17:03.267 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.155:6833/390026217 2024-09-20T06:16:50.842891+0000 2024-09-19T06:17:03.267 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6801/3684090301 2024-09-20T06:11:28.721009+0000 2024-09-19T06:17:03.267 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6800/475342876 2024-09-20T06:09:58.271775+0000 2024-09-19T06:17:03.268 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6801/475342876 2024-09-20T06:09:58.271775+0000 2024-09-19T06:17:03.268 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.155:6832/390026217 2024-09-20T06:16:50.842891+0000 2024-09-19T06:17:03.268 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/3753705881 2024-09-20T06:11:28.721009+0000 2024-09-19T06:17:03.268 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/4216592427 2024-09-20T06:09:58.271775+0000 2024-09-19T06:17:03.268 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/1618433917 2024-09-20T06:10:30.209033+0000 2024-09-19T06:17:03.268 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/623598636 2024-09-20T06:09:58.271775+0000 2024-09-19T06:17:03.268 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6800/3684090301 2024-09-20T06:11:28.721009+0000 2024-09-19T06:17:03.268 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6800/681468213 2024-09-20T06:10:30.209033+0000 2024-09-19T06:17:03.268 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/597749918 2024-09-20T06:10:30.209033+0000 2024-09-19T06:17:03.268 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/21853845 2024-09-20T06:11:28.721009+0000 2024-09-19T06:17:03.268 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/3515303955 2024-09-20T06:09:58.271775+0000 2024-09-19T06:17:03.269 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6801/681468213 2024-09-20T06:10:30.209033+0000 2024-09-19T06:17:03.269 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/1729980812 2024-09-20T06:11:28.721009+0000 2024-09-19T06:17:03.269 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/372276981 2024-09-20T06:10:30.209033+0000 2024-09-19T06:17:03.270 INFO:teuthology.orchestra.run.smithi053.stderr:listed 17 entries 2024-09-19T06:17:03.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.267+0000 7f759d2eb640 1 -- 172.21.15.53:0/3289910606 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f758006e300 msgr2=0x7f75800707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:03.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.267+0000 7f759d2eb640 1 --2- 172.21.15.53:0/3289910606 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f758006e300 0x7f75800707c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f75981990a0 tx=0x7f758c005d50 comp rx=0 tx=0).stop 2024-09-19T06:17:03.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.267+0000 7f759d2eb640 1 -- 172.21.15.53:0/3289910606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75980ffdc0 msgr2=0x7f759819d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:03.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.267+0000 7f759d2eb640 1 --2- 172.21.15.53:0/3289910606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75980ffdc0 0x7f759819d6f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f758800b420 tx=0x7f758800b8e0 comp rx=0 tx=0).stop 2024-09-19T06:17:03.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.267+0000 7f759d2eb640 1 -- 172.21.15.53:0/3289910606 shutdown_connections 2024-09-19T06:17:03.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.267+0000 7f759d2eb640 1 --2- 172.21.15.53:0/3289910606 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f758006e300 0x7f75800707c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:03.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.267+0000 7f759d2eb640 1 --2- 172.21.15.53:0/3289910606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75981069a0 0x7f759819dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:03.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.267+0000 7f759d2eb640 1 --2- 172.21.15.53:0/3289910606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75980ffdc0 0x7f759819d6f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:03.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.267+0000 7f759d2eb640 1 -- 172.21.15.53:0/3289910606 >> 172.21.15.53:0/3289910606 conn(0x7f75980fbc90 msgr2=0x7f75980fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:03.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.267+0000 7f759d2eb640 1 -- 172.21.15.53:0/3289910606 shutdown_connections 2024-09-19T06:17:03.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:03.267+0000 7f759d2eb640 1 -- 172.21.15.53:0/3289910606 wait complete. 2024-09-19T06:17:03.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:03 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/3289910606' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T06:17:03.990 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-19T06:17:03.990 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T06:17:04.008 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-19T06:17:04.071 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:03 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/3289910606' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T06:17:04.311 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:17:04.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.764+0000 7f286325f640 1 -- 172.21.15.53:0/3661996912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f285c107910 msgr2=0x7f285c107cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:04.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.764+0000 7f286325f640 1 --2- 172.21.15.53:0/3661996912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f285c107910 0x7f285c107cf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2850009880 tx=0x7f285002a4a0 comp rx=0 tx=0).stop 2024-09-19T06:17:04.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.764+0000 7f286325f640 1 -- 172.21.15.53:0/3661996912 shutdown_connections 2024-09-19T06:17:04.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.764+0000 7f286325f640 1 --2- 172.21.15.53:0/3661996912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f285c101910 0x7f285c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:04.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.764+0000 7f286325f640 1 --2- 172.21.15.53:0/3661996912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f285c107910 0x7f285c107cf0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:04.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.764+0000 7f286325f640 1 -- 172.21.15.53:0/3661996912 >> 172.21.15.53:0/3661996912 conn(0x7f285c0fd630 msgr2=0x7f285c0ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:04.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.764+0000 7f286325f640 1 -- 172.21.15.53:0/3661996912 shutdown_connections 2024-09-19T06:17:04.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.764+0000 7f286325f640 1 -- 172.21.15.53:0/3661996912 wait complete. 2024-09-19T06:17:04.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.764+0000 7f286325f640 1 Processor -- start 2024-09-19T06:17:04.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.764+0000 7f286325f640 1 -- start start 2024-09-19T06:17:04.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.765+0000 7f286325f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f285c101910 0x7f285c19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:04.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.765+0000 7f286325f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f285c107910 0x7f285c19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:04.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.765+0000 7f286325f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f285c10ed50 con 0x7f285c101910 2024-09-19T06:17:04.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.765+0000 7f286325f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f285c10ebd0 con 0x7f285c107910 2024-09-19T06:17:04.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.765+0000 7f2860ffb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f285c101910 0x7f285c19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:04.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.765+0000 7f2860ffb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f285c101910 0x7f285c19d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50592/0 (socket says 172.21.15.53:50592) 2024-09-19T06:17:04.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.765+0000 7f2860ffb640 1 -- 172.21.15.53:0/2700070274 learned_addr learned my addr 172.21.15.53:0/2700070274 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:17:04.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.765+0000 7f285bfff640 1 --2- 172.21.15.53:0/2700070274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f285c107910 0x7f285c19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:04.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.765+0000 7f2860ffb640 1 -- 172.21.15.53:0/2700070274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f285c107910 msgr2=0x7f285c19db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:04.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.765+0000 7f2860ffb640 1 --2- 172.21.15.53:0/2700070274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f285c107910 0x7f285c19db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:04.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.765+0000 7f2860ffb640 1 -- 172.21.15.53:0/2700070274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f285c197710 con 0x7f285c101910 2024-09-19T06:17:04.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.765+0000 7f2860ffb640 1 --2- 172.21.15.53:0/2700070274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f285c101910 0x7f285c19d640 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f285002a9c0 tx=0x7f28500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:04.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.765+0000 7f2859ffb640 1 -- 172.21.15.53:0/2700070274 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f285003f070 con 0x7f285c101910 2024-09-19T06:17:04.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.765+0000 7f2859ffb640 1 -- 172.21.15.53:0/2700070274 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2850004ac0 con 0x7f285c101910 2024-09-19T06:17:04.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.766+0000 7f2859ffb640 1 -- 172.21.15.53:0/2700070274 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2850035030 con 0x7f285c101910 2024-09-19T06:17:04.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.766+0000 7f286325f640 1 -- 172.21.15.53:0/2700070274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f285c1979a0 con 0x7f285c101910 2024-09-19T06:17:04.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.766+0000 7f286325f640 1 -- 172.21.15.53:0/2700070274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f285c197ed0 con 0x7f285c101910 2024-09-19T06:17:04.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.767+0000 7f286325f640 1 -- 172.21.15.53:0/2700070274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f285c103050 con 0x7f285c101910 2024-09-19T06:17:04.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.768+0000 7f2859ffb640 1 -- 172.21.15.53:0/2700070274 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f2850042070 con 0x7f285c101910 2024-09-19T06:17:04.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.768+0000 7f2859ffb640 1 --2- 172.21.15.53:0/2700070274 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f284c06e310 0x7f284c0707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:04.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.769+0000 7f2859ffb640 1 -- 172.21.15.53:0/2700070274 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7224+0+0 (secure 0 0 0) 0x7f28500af3d0 con 0x7f285c101910 2024-09-19T06:17:04.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.769+0000 7f285bfff640 1 --2- 172.21.15.53:0/2700070274 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f284c06e310 0x7f284c0707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:04.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.769+0000 7f285bfff640 1 --2- 172.21.15.53:0/2700070274 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f284c06e310 0x7f284c0707d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f285c198f60 tx=0x7f284800b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:04.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.773+0000 7f2859ffb640 1 -- 172.21.15.53:0/2700070274 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f28500801d0 con 0x7f285c101910 2024-09-19T06:17:04.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.965+0000 7f286325f640 1 -- 172.21.15.53:0/2700070274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f285c109e80 con 0x7f285c101910 2024-09-19T06:17:04.968 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:04 smithi053 ceph-mon[27968]: pgmap v225: 129 pgs: 129 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 6.9 KiB/s rd, 171 B/s wr, 6 op/s 2024-09-19T06:17:04.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.966+0000 7f2859ffb640 1 -- 172.21.15.53:0/2700070274 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v70) ==== 81+0+1001 (secure 0 0 0) 0x7f2850084a00 con 0x7f285c101910 2024-09-19T06:17:04.969 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.155:6833/390026217 2024-09-20T06:16:50.842891+0000 2024-09-19T06:17:04.969 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6801/3684090301 2024-09-20T06:11:28.721009+0000 2024-09-19T06:17:04.969 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6800/475342876 2024-09-20T06:09:58.271775+0000 2024-09-19T06:17:04.969 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6801/475342876 2024-09-20T06:09:58.271775+0000 2024-09-19T06:17:04.969 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.155:6832/390026217 2024-09-20T06:16:50.842891+0000 2024-09-19T06:17:04.969 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/3753705881 2024-09-20T06:11:28.721009+0000 2024-09-19T06:17:04.969 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/4216592427 2024-09-20T06:09:58.271775+0000 2024-09-19T06:17:04.969 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/1618433917 2024-09-20T06:10:30.209033+0000 2024-09-19T06:17:04.969 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/623598636 2024-09-20T06:09:58.271775+0000 2024-09-19T06:17:04.970 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6800/3684090301 2024-09-20T06:11:28.721009+0000 2024-09-19T06:17:04.970 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6800/681468213 2024-09-20T06:10:30.209033+0000 2024-09-19T06:17:04.970 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/597749918 2024-09-20T06:10:30.209033+0000 2024-09-19T06:17:04.970 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/21853845 2024-09-20T06:11:28.721009+0000 2024-09-19T06:17:04.970 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/3515303955 2024-09-20T06:09:58.271775+0000 2024-09-19T06:17:04.970 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6801/681468213 2024-09-20T06:10:30.209033+0000 2024-09-19T06:17:04.970 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/1729980812 2024-09-20T06:11:28.721009+0000 2024-09-19T06:17:04.970 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/372276981 2024-09-20T06:10:30.209033+0000 2024-09-19T06:17:04.971 INFO:teuthology.orchestra.run.smithi053.stderr:listed 17 entries 2024-09-19T06:17:04.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.968+0000 7f286325f640 1 -- 172.21.15.53:0/2700070274 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f284c06e310 msgr2=0x7f284c0707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:04.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.968+0000 7f286325f640 1 --2- 172.21.15.53:0/2700070274 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f284c06e310 0x7f284c0707d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f285c198f60 tx=0x7f284800b390 comp rx=0 tx=0).stop 2024-09-19T06:17:04.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.968+0000 7f286325f640 1 -- 172.21.15.53:0/2700070274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f285c101910 msgr2=0x7f285c19d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:04.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.968+0000 7f286325f640 1 --2- 172.21.15.53:0/2700070274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f285c101910 0x7f285c19d640 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f285002a9c0 tx=0x7f28500046c0 comp rx=0 tx=0).stop 2024-09-19T06:17:04.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.968+0000 7f286325f640 1 -- 172.21.15.53:0/2700070274 shutdown_connections 2024-09-19T06:17:04.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.968+0000 7f286325f640 1 --2- 172.21.15.53:0/2700070274 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f284c06e310 0x7f284c0707d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:04.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.968+0000 7f286325f640 1 --2- 172.21.15.53:0/2700070274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f285c107910 0x7f285c19db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:04.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.968+0000 7f286325f640 1 --2- 172.21.15.53:0/2700070274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f285c101910 0x7f285c19d640 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:04.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.968+0000 7f286325f640 1 -- 172.21.15.53:0/2700070274 >> 172.21.15.53:0/2700070274 conn(0x7f285c0fd630 msgr2=0x7f285c0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:04.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.968+0000 7f286325f640 1 -- 172.21.15.53:0/2700070274 shutdown_connections 2024-09-19T06:17:04.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:04.969+0000 7f286325f640 1 -- 172.21.15.53:0/2700070274 wait complete. 2024-09-19T06:17:05.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:04 smithi155 ceph-mon[35292]: pgmap v225: 129 pgs: 129 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 6.9 KiB/s rd, 171 B/s wr, 6 op/s 2024-09-19T06:17:05.737 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi053.front.sepia.ceph.com... 2024-09-19T06:17:05.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:17:05.737 DEBUG:teuthology.orchestra.run.smithi053:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T06:17:05.805 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi053.stderr:ceph-fuse[78081]: fuse finished with error 0 and tester_r 0 2024-09-19T06:17:05.806 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T06:17:06.004 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:05 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/2700070274' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T06:17:06.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:05 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/2700070274' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T06:17:07.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:06 smithi053 ceph-mon[27968]: pgmap v226: 129 pgs: 129 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 6.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-19T06:17:07.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:06 smithi155 ceph-mon[35292]: pgmap v226: 129 pgs: 129 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 6.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-19T06:17:08.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:07 smithi053 ceph-mon[27968]: pgmap v227: 129 pgs: 129 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-19T06:17:08.134 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:07 smithi155 ceph-mon[35292]: pgmap v227: 129 pgs: 129 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-19T06:17:10.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:09 smithi053 ceph-mon[27968]: pgmap v228: 129 pgs: 129 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 4.8 KiB/s rd, 341 B/s wr, 4 op/s 2024-09-19T06:17:10.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:09 smithi155 ceph-mon[35292]: pgmap v228: 129 pgs: 129 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 4.8 KiB/s rd, 341 B/s wr, 4 op/s 2024-09-19T06:17:11.908 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2024-09-19T06:17:11.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:17:11.909 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T06:17:11.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:17:11.937 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2024-09-19T06:17:11.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:17:11.958 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link delete ceph-brx 2024-09-19T06:17:12.032 INFO:teuthology.orchestra.run.smithi155.stderr:Cannot find device "ceph-brx" 2024-09-19T06:17:12.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T06:17:12.034 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-19T06:17:12.034 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-19T06:17:12.034 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-19T06:17:12.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:11 smithi053 ceph-mon[27968]: pgmap v229: 129 pgs: 129 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 341 B/s wr, 0 op/s 2024-09-19T06:17:12.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:11 smithi155 ceph-mon[35292]: pgmap v229: 129 pgs: 129 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 341 B/s wr, 0 op/s 2024-09-19T06:17:12.310 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:17:12.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.733+0000 7f6d3e597640 1 -- 172.21.15.53:0/1708814341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d38106800 msgr2=0x7f6d38106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:12.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.733+0000 7f6d3e597640 1 --2- 172.21.15.53:0/1708814341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d38106800 0x7f6d38106be0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6d28009880 tx=0x7f6d2802a4a0 comp rx=0 tx=0).stop 2024-09-19T06:17:12.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.733+0000 7f6d3e597640 1 -- 172.21.15.53:0/1708814341 shutdown_connections 2024-09-19T06:17:12.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.733+0000 7f6d3e597640 1 --2- 172.21.15.53:0/1708814341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d38100800 0x7f6d38100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:12.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.733+0000 7f6d3e597640 1 --2- 172.21.15.53:0/1708814341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d38106800 0x7f6d38106be0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:12.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.733+0000 7f6d3e597640 1 -- 172.21.15.53:0/1708814341 >> 172.21.15.53:0/1708814341 conn(0x7f6d380fc520 msgr2=0x7f6d380fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:12.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.733+0000 7f6d3e597640 1 -- 172.21.15.53:0/1708814341 shutdown_connections 2024-09-19T06:17:12.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.733+0000 7f6d3e597640 1 -- 172.21.15.53:0/1708814341 wait complete. 2024-09-19T06:17:12.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.734+0000 7f6d3e597640 1 Processor -- start 2024-09-19T06:17:12.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.734+0000 7f6d3e597640 1 -- start start 2024-09-19T06:17:12.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.734+0000 7f6d3e597640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d38100800 0x7f6d3819d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:12.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.734+0000 7f6d3e597640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d38106800 0x7f6d3819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:12.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.735+0000 7f6d3e597640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3810dca0 con 0x7f6d38100800 2024-09-19T06:17:12.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.735+0000 7f6d3e597640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3810db20 con 0x7f6d38106800 2024-09-19T06:17:12.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.735+0000 7f6d37fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d38100800 0x7f6d3819d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:12.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.735+0000 7f6d37fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d38100800 0x7f6d3819d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42998/0 (socket says 172.21.15.53:42998) 2024-09-19T06:17:12.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.735+0000 7f6d37fff640 1 -- 172.21.15.53:0/315132837 learned_addr learned my addr 172.21.15.53:0/315132837 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:17:12.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.735+0000 7f6d377fe640 1 --2- 172.21.15.53:0/315132837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d38106800 0x7f6d3819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:12.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.735+0000 7f6d37fff640 1 -- 172.21.15.53:0/315132837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d38106800 msgr2=0x7f6d3819dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:12.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.735+0000 7f6d37fff640 1 --2- 172.21.15.53:0/315132837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d38106800 0x7f6d3819dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:12.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.735+0000 7f6d37fff640 1 -- 172.21.15.53:0/315132837 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d38197800 con 0x7f6d38100800 2024-09-19T06:17:12.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.735+0000 7f6d377fe640 1 --2- 172.21.15.53:0/315132837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d38106800 0x7f6d3819dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:17:12.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.735+0000 7f6d37fff640 1 --2- 172.21.15.53:0/315132837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d38100800 0x7f6d3819d680 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6d28006010 tx=0x7f6d280048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:12.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.735+0000 7f6d357fa640 1 -- 172.21.15.53:0/315132837 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6d2803f070 con 0x7f6d38100800 2024-09-19T06:17:12.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.735+0000 7f6d357fa640 1 -- 172.21.15.53:0/315132837 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d28003aa0 con 0x7f6d38100800 2024-09-19T06:17:12.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.735+0000 7f6d357fa640 1 -- 172.21.15.53:0/315132837 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6d28035030 con 0x7f6d38100800 2024-09-19T06:17:12.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.735+0000 7f6d3e597640 1 -- 172.21.15.53:0/315132837 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d38197a90 con 0x7f6d38100800 2024-09-19T06:17:12.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.736+0000 7f6d3e597640 1 -- 172.21.15.53:0/315132837 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d38197fc0 con 0x7f6d38100800 2024-09-19T06:17:12.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.737+0000 7f6d1effd640 1 -- 172.21.15.53:0/315132837 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d18005290 con 0x7f6d38100800 2024-09-19T06:17:12.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.737+0000 7f6d357fa640 1 -- 172.21.15.53:0/315132837 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f6d28042070 con 0x7f6d38100800 2024-09-19T06:17:12.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.738+0000 7f6d357fa640 1 --2- 172.21.15.53:0/315132837 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f6d2006e2b0 0x7f6d20070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:12.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.738+0000 7f6d357fa640 1 -- 172.21.15.53:0/315132837 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7224+0+0 (secure 0 0 0) 0x7f6d280af4e0 con 0x7f6d38100800 2024-09-19T06:17:12.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.738+0000 7f6d377fe640 1 --2- 172.21.15.53:0/315132837 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f6d2006e2b0 0x7f6d20070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:12.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.738+0000 7f6d377fe640 1 --2- 172.21.15.53:0/315132837 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f6d2006e2b0 0x7f6d20070770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c006fd0 tx=0x7f6d2c008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:12.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.742+0000 7f6d357fa640 1 -- 172.21.15.53:0/315132837 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6d2807c630 con 0x7f6d38100800 2024-09-19T06:17:12.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.957+0000 7f6d1effd640 1 -- 172.21.15.53:0/315132837 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6d18005d90 con 0x7f6d38100800 2024-09-19T06:17:12.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.957+0000 7f6d357fa640 1 -- 172.21.15.53:0/315132837 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) ==== 53+0+148 (secure 0 0 0) 0x7f6d280802e0 con 0x7f6d38100800 2024-09-19T06:17:12.959 INFO:teuthology.orchestra.run.smithi053.stdout:name: foo, metadata pool: cephfs.foo.meta, data pools: [cephfs.foo.data ] 2024-09-19T06:17:12.959 INFO:teuthology.orchestra.run.smithi053.stdout:name: bar, metadata pool: cephfs.bar.meta, data pools: [cephfs.bar.data ] 2024-09-19T06:17:12.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.959+0000 7f6d1effd640 1 -- 172.21.15.53:0/315132837 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f6d2006e2b0 msgr2=0x7f6d20070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:12.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.959+0000 7f6d1effd640 1 --2- 172.21.15.53:0/315132837 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f6d2006e2b0 0x7f6d20070770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c006fd0 tx=0x7f6d2c008040 comp rx=0 tx=0).stop 2024-09-19T06:17:12.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.959+0000 7f6d1effd640 1 -- 172.21.15.53:0/315132837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d38100800 msgr2=0x7f6d3819d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:12.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.959+0000 7f6d1effd640 1 --2- 172.21.15.53:0/315132837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d38100800 0x7f6d3819d680 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6d28006010 tx=0x7f6d280048f0 comp rx=0 tx=0).stop 2024-09-19T06:17:12.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.959+0000 7f6d1effd640 1 -- 172.21.15.53:0/315132837 shutdown_connections 2024-09-19T06:17:12.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.959+0000 7f6d1effd640 1 --2- 172.21.15.53:0/315132837 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f6d2006e2b0 0x7f6d20070770 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:12.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.959+0000 7f6d1effd640 1 --2- 172.21.15.53:0/315132837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d38106800 0x7f6d3819dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:12.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.959+0000 7f6d1effd640 1 --2- 172.21.15.53:0/315132837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d38100800 0x7f6d3819d680 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:12.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.959+0000 7f6d1effd640 1 -- 172.21.15.53:0/315132837 >> 172.21.15.53:0/315132837 conn(0x7f6d380fc520 msgr2=0x7f6d3806a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:12.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.959+0000 7f6d1effd640 1 -- 172.21.15.53:0/315132837 shutdown_connections 2024-09-19T06:17:12.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:17:12.959+0000 7f6d1effd640 1 -- 172.21.15.53:0/315132837 wait complete. 2024-09-19T06:17:13.744 INFO:tasks.cephfs.mount:Mounting Ceph FS bar; just confirmed its presence on cluster 2024-09-19T06:17:13.744 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T06:17:13.744 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi155.front.sepia.ceph.com 2024-09-19T06:17:13.744 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-19T06:17:13.744 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-19T06:17:13.744 INFO:tasks.cephfs.mount:self.cephfs_name = bar 2024-09-19T06:17:13.744 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T06:17:13.744 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T06:17:13.744 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T06:17:13.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:17:13.745 DEBUG:teuthology.orchestra.run.smithi155:> ip addr 2024-09-19T06:17:13.764 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T06:17:13.764 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T06:17:13.764 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T06:17:13.764 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-09-19T06:17:13.764 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2024-09-19T06:17:13.764 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-09-19T06:17:13.764 INFO:teuthology.orchestra.run.smithi155.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T06:17:13.765 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fa brd ff:ff:ff:ff:ff:ff 2024-09-19T06:17:13.765 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f0 2024-09-19T06:17:13.765 INFO:teuthology.orchestra.run.smithi155.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T06:17:13.765 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fb brd ff:ff:ff:ff:ff:ff 2024-09-19T06:17:13.765 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f1 2024-09-19T06:17:13.765 INFO:teuthology.orchestra.run.smithi155.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T06:17:13.765 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ec brd ff:ff:ff:ff:ff:ff 2024-09-19T06:17:13.765 INFO:teuthology.orchestra.run.smithi155.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T06:17:13.765 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ed brd ff:ff:ff:ff:ff:ff 2024-09-19T06:17:13.765 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.21.15.155/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-19T06:17:13.765 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft 42300sec preferred_lft 42300sec 2024-09-19T06:17:13.765 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::ec4:7aff:fe88:6ded/64 scope link noprefixroute 2024-09-19T06:17:13.765 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-09-19T06:17:13.766 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-19T06:17:13.766 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T06:17:13.766 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-09-19T06:17:13.766 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link add name ceph-brx type bridge 2024-09-19T06:17:13.766 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip addr flush dev ceph-brx 2024-09-19T06:17:13.766 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set ceph-brx up 2024-09-19T06:17:13.766 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-19T06:17:13.766 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-09-19T06:17:13.860 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:17:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T06:17:13.974 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:13 smithi155 ceph-mon[35292]: pgmap v230: 129 pgs: 129 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 341 B/s wr, 0 op/s 2024-09-19T06:17:13.975 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:13 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/315132837' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T06:17:13.975 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:13 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:17:14.009 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:17:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:17:14.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:17:14.016 DEBUG:teuthology.orchestra.run.smithi155:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T06:17:14.043 INFO:teuthology.orchestra.run.smithi155.stdout:1 2024-09-19T06:17:14.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:17:14.045 DEBUG:teuthology.orchestra.run.smithi155:> route 2024-09-19T06:17:14.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:13 smithi053 ceph-mon[27968]: pgmap v230: 129 pgs: 129 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 341 B/s wr, 0 op/s 2024-09-19T06:17:14.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:13 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/315132837' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T06:17:14.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:13 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:17:14.110 INFO:teuthology.orchestra.run.smithi155.stdout:Kernel IP routing table 2024-09-19T06:17:14.111 INFO:teuthology.orchestra.run.smithi155.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T06:17:14.111 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T06:17:14.111 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T06:17:14.111 INFO:teuthology.orchestra.run.smithi155.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T06:17:14.111 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T06:17:14.111 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-09-19T06:17:14.111 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T06:17:14.111 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T06:17:14.112 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T06:17:14.112 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-09-19T06:17:14.199 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:17:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T06:17:14.531 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:17:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:17:14.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:17:14.538 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2024-09-19T06:17:14.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:17:14.556 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list-id 2024-09-19T06:17:14.609 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T06:17:14.609 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-09-19T06:17:14.609 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T06:17:14.609 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-19T06:17:14.609 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-09-19T06:17:14.697 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:17:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T06:17:14.720 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:17:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:17:14.726 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-19T06:17:14.726 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T06:17:14.726 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-09-19T06:17:14.726 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-19T06:17:14.726 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-19T06:17:14.726 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-19T06:17:14.727 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-19T06:17:14.727 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-19T06:17:14.727 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-09-19T06:17:14.775 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:17:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T06:17:14.847 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:17:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:17:14.853 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T06:17:14.853 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-09-19T06:17:14.853 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set brx.0 up 2024-09-19T06:17:14.853 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set dev brx.0 master ceph-brx 2024-09-19T06:17:14.853 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-09-19T06:17:14.901 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:17:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T06:17:14.925 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:17:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:17:14.931 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {'cephfs_name': 'bar'} 2024-09-19T06:17:14.931 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T06:17:14.931 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-19T06:17:14.946 INFO:teuthology.orchestra.run.smithi155.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-19T06:17:14.946 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T06:17:14.946 DEBUG:teuthology.orchestra.run.smithi155:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-19T06:17:15.002 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2024-09-19T06:17:15.068 DEBUG:teuthology.orchestra.run.smithi155:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T06:17:15.124 INFO:teuthology.orchestra.run.smithi155.stdout:/proc 2024-09-19T06:17:15.124 INFO:teuthology.orchestra.run.smithi155.stdout:/sys 2024-09-19T06:17:15.124 INFO:teuthology.orchestra.run.smithi155.stdout:/dev 2024-09-19T06:17:15.125 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/security 2024-09-19T06:17:15.125 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/shm 2024-09-19T06:17:15.125 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/pts 2024-09-19T06:17:15.125 INFO:teuthology.orchestra.run.smithi155.stdout:/run 2024-09-19T06:17:15.125 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/cgroup 2024-09-19T06:17:15.125 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/pstore 2024-09-19T06:17:15.125 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/bpf 2024-09-19T06:17:15.125 INFO:teuthology.orchestra.run.smithi155.stdout:/ 2024-09-19T06:17:15.125 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/selinux 2024-09-19T06:17:15.125 INFO:teuthology.orchestra.run.smithi155.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T06:17:15.125 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/mqueue 2024-09-19T06:17:15.125 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/hugepages 2024-09-19T06:17:15.125 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/debug 2024-09-19T06:17:15.125 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/tracing 2024-09-19T06:17:15.126 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/fuse/connections 2024-09-19T06:17:15.126 INFO:teuthology.orchestra.run.smithi155.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T06:17:15.126 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/config 2024-09-19T06:17:15.126 INFO:teuthology.orchestra.run.smithi155.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T06:17:15.126 INFO:teuthology.orchestra.run.smithi155.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T06:17:15.126 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T06:17:15.126 INFO:teuthology.orchestra.run.smithi155.stdout:/run/user/1000 2024-09-19T06:17:15.126 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph 2024-09-19T06:17:15.126 INFO:teuthology.orchestra.run.smithi155.stdout:/run/user/0 2024-09-19T06:17:15.126 INFO:teuthology.orchestra.run.smithi155.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T06:17:15.126 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/containers/storage/overlay 2024-09-19T06:17:15.126 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/containers/storage/overlay/0c23a7363c2fa109e2277f871b1129377da825a0b2c07e059adc2536745fd9be/merged 2024-09-19T06:17:15.126 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/containers/storage/overlay/c2bf72fa62a5d5104600ca8e52e325aefb16c2a68c4e3f941a92222d84080599/merged 2024-09-19T06:17:15.126 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/containers/storage/overlay/87162474e0100a33ba28cbf7b94c2ebc3f295d56ee1b36bf3ace5e60b09b864b/merged 2024-09-19T06:17:15.127 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/containers/storage/overlay/4d78e2e628e313adb202afdf82da0fdab46a190c1160b3237f805931d9b8179e/merged 2024-09-19T06:17:15.127 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/containers/storage/overlay/f6595e6e9aea1df64267cb51eac08e3c98f238e786962d5b7493bfc75f0f9af4/merged 2024-09-19T06:17:15.127 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/containers/storage/overlay/7d7df90ea3c0bde704426e722e9976d7bf89ccdc751fb51f2bc988f09c6797a0/merged 2024-09-19T06:17:15.127 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/containers/storage/overlay/b7a542bb3c11083c5ec934498137f6e492baaa85cc70e0b6772c8d00523148e3/merged 2024-09-19T06:17:15.127 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/containers/storage/overlay/96f51b202db3cae968643566605ee9d93f80e11be6d3e2688bfdc3755c8bf48c/merged 2024-09-19T06:17:15.127 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/containers/storage/overlay/4ba24e8d60e3fce17e725e46ccfa7eab9f3177311c683f0741630b821c9a3495/merged 2024-09-19T06:17:15.127 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/containers/storage/overlay/4850826bf2a4f2752ec5c307570108fcb89c87ec7b90e1cd1e16bf45d99fa860/merged 2024-09-19T06:17:15.127 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/containers/storage/overlay/0a144b2ef7caf6677eda53af8ba063a240e0e238ad327ba03f7d51800f4830a5/merged 2024-09-19T06:17:15.127 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns 2024-09-19T06:17:15.127 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T06:17:15.127 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T06:17:15.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:17:15.128 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2024-09-19T06:17:15.181 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T06:17:15.181 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1 --client_fs=bar) 2024-09-19T06:17:15.223 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2024-09-19T06:17:15.248 DEBUG:teuthology.orchestra.run.smithi155:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T06:17:15.304 INFO:teuthology.orchestra.run.smithi155.stdout:/proc 2024-09-19T06:17:15.305 INFO:teuthology.orchestra.run.smithi155.stdout:/sys 2024-09-19T06:17:15.305 INFO:teuthology.orchestra.run.smithi155.stdout:/dev 2024-09-19T06:17:15.305 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/security 2024-09-19T06:17:15.305 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/shm 2024-09-19T06:17:15.305 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/pts 2024-09-19T06:17:15.305 INFO:teuthology.orchestra.run.smithi155.stdout:/run 2024-09-19T06:17:15.305 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/cgroup 2024-09-19T06:17:15.305 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/pstore 2024-09-19T06:17:15.305 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/bpf 2024-09-19T06:17:15.305 INFO:teuthology.orchestra.run.smithi155.stdout:/ 2024-09-19T06:17:15.305 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/selinux 2024-09-19T06:17:15.305 INFO:teuthology.orchestra.run.smithi155.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T06:17:15.305 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/mqueue 2024-09-19T06:17:15.306 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/hugepages 2024-09-19T06:17:15.306 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/debug 2024-09-19T06:17:15.306 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/tracing 2024-09-19T06:17:15.306 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/fuse/connections 2024-09-19T06:17:15.306 INFO:teuthology.orchestra.run.smithi155.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T06:17:15.306 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/config 2024-09-19T06:17:15.306 INFO:teuthology.orchestra.run.smithi155.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T06:17:15.306 INFO:teuthology.orchestra.run.smithi155.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T06:17:15.306 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T06:17:15.306 INFO:teuthology.orchestra.run.smithi155.stdout:/run/user/1000 2024-09-19T06:17:15.306 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph 2024-09-19T06:17:15.306 INFO:teuthology.orchestra.run.smithi155.stdout:/run/user/0 2024-09-19T06:17:15.306 INFO:teuthology.orchestra.run.smithi155.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T06:17:15.307 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/containers/storage/overlay 2024-09-19T06:17:15.307 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/containers/storage/overlay/0c23a7363c2fa109e2277f871b1129377da825a0b2c07e059adc2536745fd9be/merged 2024-09-19T06:17:15.307 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/containers/storage/overlay/c2bf72fa62a5d5104600ca8e52e325aefb16c2a68c4e3f941a92222d84080599/merged 2024-09-19T06:17:15.307 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/containers/storage/overlay/87162474e0100a33ba28cbf7b94c2ebc3f295d56ee1b36bf3ace5e60b09b864b/merged 2024-09-19T06:17:15.307 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/containers/storage/overlay/4d78e2e628e313adb202afdf82da0fdab46a190c1160b3237f805931d9b8179e/merged 2024-09-19T06:17:15.307 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/containers/storage/overlay/f6595e6e9aea1df64267cb51eac08e3c98f238e786962d5b7493bfc75f0f9af4/merged 2024-09-19T06:17:15.307 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/containers/storage/overlay/7d7df90ea3c0bde704426e722e9976d7bf89ccdc751fb51f2bc988f09c6797a0/merged 2024-09-19T06:17:15.307 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/containers/storage/overlay/b7a542bb3c11083c5ec934498137f6e492baaa85cc70e0b6772c8d00523148e3/merged 2024-09-19T06:17:15.307 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/containers/storage/overlay/96f51b202db3cae968643566605ee9d93f80e11be6d3e2688bfdc3755c8bf48c/merged 2024-09-19T06:17:15.307 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/containers/storage/overlay/4ba24e8d60e3fce17e725e46ccfa7eab9f3177311c683f0741630b821c9a3495/merged 2024-09-19T06:17:15.307 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/containers/storage/overlay/4850826bf2a4f2752ec5c307570108fcb89c87ec7b90e1cd1e16bf45d99fa860/merged 2024-09-19T06:17:15.307 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/containers/storage/overlay/0a144b2ef7caf6677eda53af8ba063a240e0e238ad327ba03f7d51800f4830a5/merged 2024-09-19T06:17:15.307 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns 2024-09-19T06:17:15.307 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T06:17:15.308 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T06:17:15.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:17:15.308 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2024-09-19T06:17:15.334 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:2024-09-19T06:17:15.332+0000 7f1aad9e8480 -1 init, newargv = 0x55fec357e120 newargc=15 2024-09-19T06:17:15.334 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[64780]: starting ceph client 2024-09-19T06:17:15.339 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[64780]: starting fuse 2024-09-19T06:17:15.349 INFO:teuthology.orchestra.run.smithi155.stdout:132 2024-09-19T06:17:15.349 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [132] 2024-09-19T06:17:15.350 DEBUG:teuthology.orchestra.run.smithi155:> sudo stdin-killer -- python3 -c ' 2024-09-19T06:17:15.350 DEBUG:teuthology.orchestra.run.smithi155:> import glob 2024-09-19T06:17:15.350 DEBUG:teuthology.orchestra.run.smithi155:> import re 2024-09-19T06:17:15.350 DEBUG:teuthology.orchestra.run.smithi155:> import os 2024-09-19T06:17:15.350 DEBUG:teuthology.orchestra.run.smithi155:> import subprocess 2024-09-19T06:17:15.350 DEBUG:teuthology.orchestra.run.smithi155:> 2024-09-19T06:17:15.350 DEBUG:teuthology.orchestra.run.smithi155:> def _find_admin_socket(client_name): 2024-09-19T06:17:15.350 DEBUG:teuthology.orchestra.run.smithi155:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-19T06:17:15.350 DEBUG:teuthology.orchestra.run.smithi155:> files = glob.glob(asok_path) 2024-09-19T06:17:15.350 DEBUG:teuthology.orchestra.run.smithi155:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-19T06:17:15.350 DEBUG:teuthology.orchestra.run.smithi155:> 2024-09-19T06:17:15.350 DEBUG:teuthology.orchestra.run.smithi155:> # Given a non-glob path, it better be there 2024-09-19T06:17:15.351 DEBUG:teuthology.orchestra.run.smithi155:> if "*" not in asok_path: 2024-09-19T06:17:15.351 DEBUG:teuthology.orchestra.run.smithi155:> assert(len(files) == 1) 2024-09-19T06:17:15.351 DEBUG:teuthology.orchestra.run.smithi155:> return files[0] 2024-09-19T06:17:15.351 DEBUG:teuthology.orchestra.run.smithi155:> 2024-09-19T06:17:15.351 DEBUG:teuthology.orchestra.run.smithi155:> for f in files: 2024-09-19T06:17:15.351 DEBUG:teuthology.orchestra.run.smithi155:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T06:17:15.351 DEBUG:teuthology.orchestra.run.smithi155:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T06:17:15.351 DEBUG:teuthology.orchestra.run.smithi155:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T06:17:15.351 DEBUG:teuthology.orchestra.run.smithi155:> contents = proc_f.read() 2024-09-19T06:17:15.351 DEBUG:teuthology.orchestra.run.smithi155:> if mountpoint in contents: 2024-09-19T06:17:15.351 DEBUG:teuthology.orchestra.run.smithi155:> return f 2024-09-19T06:17:15.351 DEBUG:teuthology.orchestra.run.smithi155:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T06:17:15.351 DEBUG:teuthology.orchestra.run.smithi155:> 2024-09-19T06:17:15.351 DEBUG:teuthology.orchestra.run.smithi155:> print(_find_admin_socket("client.1")) 2024-09-19T06:17:15.351 DEBUG:teuthology.orchestra.run.smithi155:> ' 2024-09-19T06:17:15.467 INFO:teuthology.orchestra.run.smithi155.stdout:/var/run/ceph/ceph-client.1.64780.asok 2024-09-19T06:17:15.470 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-19T06:17:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:17:15.477 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.64780.asok 2024-09-19T06:17:15.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:17:15.478 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.64780.asok status 2024-09-19T06:17:15.611 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-19T06:17:15.611 INFO:teuthology.orchestra.run.smithi155.stdout: "metadata": { 2024-09-19T06:17:15.611 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T06:17:15.611 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T06:17:15.612 INFO:teuthology.orchestra.run.smithi155.stdout: "entity_id": "1", 2024-09-19T06:17:15.612 INFO:teuthology.orchestra.run.smithi155.stdout: "hostname": "smithi155", 2024-09-19T06:17:15.612 INFO:teuthology.orchestra.run.smithi155.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-19T06:17:15.612 INFO:teuthology.orchestra.run.smithi155.stdout: "pid": "64780", 2024-09-19T06:17:15.612 INFO:teuthology.orchestra.run.smithi155.stdout: "root": "/" 2024-09-19T06:17:15.612 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-19T06:17:15.612 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_count": 0, 2024-09-19T06:17:15.612 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_pinned_count": 0, 2024-09-19T06:17:15.612 INFO:teuthology.orchestra.run.smithi155.stdout: "id": 24375, 2024-09-19T06:17:15.612 INFO:teuthology.orchestra.run.smithi155.stdout: "inst": { 2024-09-19T06:17:15.612 INFO:teuthology.orchestra.run.smithi155.stdout: "name": { 2024-09-19T06:17:15.612 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "client", 2024-09-19T06:17:15.612 INFO:teuthology.orchestra.run.smithi155.stdout: "num": 24375 2024-09-19T06:17:15.612 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-19T06:17:15.613 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2024-09-19T06:17:15.613 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2024-09-19T06:17:15.613 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2024-09-19T06:17:15.613 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 2794880069 2024-09-19T06:17:15.613 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-19T06:17:15.613 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-19T06:17:15.613 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2024-09-19T06:17:15.613 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2024-09-19T06:17:15.613 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2024-09-19T06:17:15.613 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 2794880069 2024-09-19T06:17:15.613 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-19T06:17:15.613 INFO:teuthology.orchestra.run.smithi155.stdout: "inst_str": "client.24375 192.168.0.1:0/2794880069", 2024-09-19T06:17:15.613 INFO:teuthology.orchestra.run.smithi155.stdout: "addr_str": "192.168.0.1:0/2794880069", 2024-09-19T06:17:15.613 INFO:teuthology.orchestra.run.smithi155.stdout: "inode_count": 1, 2024-09-19T06:17:15.614 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_epoch": 13, 2024-09-19T06:17:15.614 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch": 70, 2024-09-19T06:17:15.614 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch_barrier": 0, 2024-09-19T06:17:15.614 INFO:teuthology.orchestra.run.smithi155.stdout: "blocklisted": false, 2024-09-19T06:17:15.614 INFO:teuthology.orchestra.run.smithi155.stdout: "fs_name": "bar" 2024-09-19T06:17:15.614 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-19T06:17:15.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:17:15.619 DEBUG:teuthology.orchestra.run.smithi155:> stat --file-system '--printf=%T 2024-09-19T06:17:15.619 DEBUG:teuthology.orchestra.run.smithi155:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-19T06:17:15.635 INFO:teuthology.orchestra.run.smithi155.stdout:fuseblk 2024-09-19T06:17:15.636 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-19T06:17:15.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:17:15.636 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-19T06:17:15.702 INFO:teuthology.run_tasks:Running task workunit... 2024-09-19T06:17:15.717 INFO:tasks.workunit:Pulling workunits from ref 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T06:17:15.717 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-19T06:17:15.717 INFO:tasks.workunit:timeout=3h 2024-09-19T06:17:15.718 INFO:tasks.workunit:cleanup=True 2024-09-19T06:17:15.718 DEBUG:teuthology.orchestra.run.smithi053:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-19T06:17:15.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T06:17:15.738 INFO:teuthology.orchestra.run.smithi053.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-19T06:17:15.739 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-09-19T06:17:15.798 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-09-19T06:17:15.798 DEBUG:teuthology.orchestra.run.smithi053:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-09-19T06:17:15.858 DEBUG:teuthology.orchestra.run.smithi155:> stat -- /home/ubuntu/cephtest/mnt.1 2024-09-19T06:17:15.876 INFO:teuthology.orchestra.run.smithi155.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-09-19T06:17:15.876 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-19T06:17:15.876 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 84h/132d Inode: 1 Links: 2 2024-09-19T06:17:15.876 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:17:15.876 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:fusefs_t:s0 2024-09-19T06:17:15.877 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-09-19T06:17:15.877 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-09-19 06:16:39.473877356 +0000 2024-09-19T06:17:15.877 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-09-19 06:17:15.699494523 +0000 2024-09-19T06:17:15.877 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2024-09-19T06:17:15.877 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-09-19T06:17:15.877 DEBUG:teuthology.orchestra.run.smithi155:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-09-19T06:17:15.949 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T06:17:15.949 DEBUG:teuthology.orchestra.run.smithi155:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T06:17:15.969 INFO:tasks.workunit.client.0.smithi053.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-19T06:17:15.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:15 smithi155 ceph-mon[35292]: pgmap v231: 129 pgs: 129 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 170 B/s wr, 0 op/s 2024-09-19T06:17:15.998 INFO:tasks.workunit.client.1.smithi155.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-09-19T06:17:16.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:15 smithi053 ceph-mon[27968]: pgmap v231: 129 pgs: 129 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 170 B/s wr, 0 op/s 2024-09-19T06:17:18.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:17 smithi155 ceph-mon[35292]: pgmap v232: 129 pgs: 129 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 170 B/s wr, 0 op/s 2024-09-19T06:17:18.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:17 smithi053 ceph-mon[27968]: pgmap v232: 129 pgs: 129 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 170 B/s wr, 0 op/s 2024-09-19T06:17:20.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:19 smithi155 ceph-mon[35292]: pgmap v233: 129 pgs: 129 active+clean; 586 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 597 B/s wr, 0 op/s 2024-09-19T06:17:20.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:19 smithi053 ceph-mon[27968]: pgmap v233: 129 pgs: 129 active+clean; 586 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 597 B/s wr, 0 op/s 2024-09-19T06:17:22.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:21 smithi155 ceph-mon[35292]: pgmap v234: 129 pgs: 129 active+clean; 586 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 426 B/s wr, 0 op/s 2024-09-19T06:17:22.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:21 smithi053 ceph-mon[27968]: pgmap v234: 129 pgs: 129 active+clean; 586 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 426 B/s wr, 0 op/s 2024-09-19T06:17:23.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:23 smithi155 ceph-mon[35292]: pgmap v235: 129 pgs: 129 active+clean; 586 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 511 B/s wr, 0 op/s 2024-09-19T06:17:24.346 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:23 smithi053 ceph-mon[27968]: pgmap v235: 129 pgs: 129 active+clean; 586 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 511 B/s wr, 0 op/s 2024-09-19T06:17:26.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:25 smithi155 ceph-mon[35292]: pgmap v236: 129 pgs: 129 active+clean; 586 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 511 B/s wr, 0 op/s 2024-09-19T06:17:26.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:25 smithi053 ceph-mon[27968]: pgmap v236: 129 pgs: 129 active+clean; 586 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 511 B/s wr, 0 op/s 2024-09-19T06:17:28.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:27 smithi053 ceph-mon[27968]: pgmap v237: 129 pgs: 129 active+clean; 586 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 597 B/s wr, 0 op/s 2024-09-19T06:17:28.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:27 smithi155 ceph-mon[35292]: pgmap v237: 129 pgs: 129 active+clean; 586 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 597 B/s wr, 0 op/s 2024-09-19T06:17:28.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:28 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [7, 2]}]: dispatch 2024-09-19T06:17:28.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:28 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-09-19T06:17:28.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:28 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2024-09-19T06:17:28.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:28 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-19T06:17:28.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:28 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:17:29.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:28 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [7, 2]}]: dispatch 2024-09-19T06:17:29.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:28 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-09-19T06:17:29.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:28 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2024-09-19T06:17:29.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:28 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-19T06:17:29.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:28 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:17:30.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:29 smithi155 ceph-mon[35292]: pgmap v238: 129 pgs: 129 active+clean; 587 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 767 B/s wr, 0 op/s 2024-09-19T06:17:30.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:29 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [7, 2]}]': finished 2024-09-19T06:17:30.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:29 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-09-19T06:17:30.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:29 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]': finished 2024-09-19T06:17:30.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:29 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-09-19T06:17:30.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:29 smithi155 ceph-mon[35292]: osdmap e71: 8 total, 8 up, 8 in 2024-09-19T06:17:30.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:29 smithi053 ceph-mon[27968]: pgmap v238: 129 pgs: 129 active+clean; 587 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 767 B/s wr, 0 op/s 2024-09-19T06:17:30.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [7, 2]}]': finished 2024-09-19T06:17:30.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-09-19T06:17:30.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]': finished 2024-09-19T06:17:30.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-09-19T06:17:30.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:29 smithi053 ceph-mon[27968]: osdmap e71: 8 total, 8 up, 8 in 2024-09-19T06:17:31.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:30 smithi155 ceph-mon[35292]: osdmap e72: 8 total, 8 up, 8 in 2024-09-19T06:17:31.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:30 smithi053 ceph-mon[27968]: osdmap e72: 8 total, 8 up, 8 in 2024-09-19T06:17:32.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:31 smithi155 ceph-mon[35292]: pgmap v241: 129 pgs: 2 peering, 127 active+clean; 587 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 383 B/s wr, 0 op/s 2024-09-19T06:17:32.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:31 smithi053 ceph-mon[27968]: pgmap v241: 129 pgs: 2 peering, 127 active+clean; 587 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 383 B/s wr, 0 op/s 2024-09-19T06:17:33.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:33 smithi155 ceph-mon[35292]: pgmap v242: 129 pgs: 2 peering, 127 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 383 B/s wr, 0 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T06:17:34.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:33 smithi053 ceph-mon[27968]: pgmap v242: 129 pgs: 2 peering, 127 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 383 B/s wr, 0 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T06:17:36.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:35 smithi155 ceph-mon[35292]: pgmap v243: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T06:17:36.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:35 smithi053 ceph-mon[27968]: pgmap v243: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T06:17:38.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:37 smithi053 ceph-mon[27968]: pgmap v244: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T06:17:38.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:37 smithi155 ceph-mon[35292]: pgmap v244: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T06:17:40.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:39 smithi053 ceph-mon[27968]: pgmap v245: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T06:17:40.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:39 smithi155 ceph-mon[35292]: pgmap v245: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T06:17:42.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:41 smithi155 ceph-mon[35292]: pgmap v246: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T06:17:42.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:41 smithi053 ceph-mon[27968]: pgmap v246: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T06:17:43.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:43 smithi155 ceph-mon[35292]: pgmap v247: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T06:17:43.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:43 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:17:44.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:43 smithi053 ceph-mon[27968]: pgmap v247: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T06:17:44.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:43 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:17:46.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:45 smithi155 ceph-mon[35292]: pgmap v248: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-09-19T06:17:46.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:45 smithi053 ceph-mon[27968]: pgmap v248: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-09-19T06:17:48.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:47 smithi155 ceph-mon[35292]: pgmap v249: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:17:48.357 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:47 smithi053 ceph-mon[27968]: pgmap v249: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:17:50.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:49 smithi155 ceph-mon[35292]: pgmap v250: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:17:50.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:49 smithi053 ceph-mon[27968]: pgmap v250: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:17:52.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:51 smithi155 ceph-mon[35292]: pgmap v251: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:17:52.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:51 smithi053 ceph-mon[27968]: pgmap v251: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:17:53.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:53 smithi155 ceph-mon[35292]: pgmap v252: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:17:54.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:53 smithi053 ceph-mon[27968]: pgmap v252: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:17:56.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:55 smithi155 ceph-mon[35292]: pgmap v253: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:17:56.256 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:55 smithi053 ceph-mon[27968]: pgmap v253: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:17:58.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:57 smithi155 ceph-mon[35292]: pgmap v254: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:17:58.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:57 smithi053 ceph-mon[27968]: pgmap v254: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:17:58.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:58 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:17:59.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:58 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:18:00.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:59 smithi155 ceph-mon[35292]: pgmap v255: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:18:00.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:17:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:18:00.334 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:59 smithi053 ceph-mon[27968]: pgmap v255: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:18:00.334 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:17:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:18:01.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:00 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:18:01.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:00 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:18:01.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:00 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:18:01.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:00 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:18:01.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:00 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:18:01.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:00 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:18:02.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:02 smithi053 ceph-mon[27968]: pgmap v256: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:18:02.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:02 smithi155 ceph-mon[35292]: pgmap v256: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:18:04.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:04 smithi053 ceph-mon[27968]: pgmap v257: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:18:04.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:04 smithi155 ceph-mon[35292]: pgmap v257: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:18:06.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:06 smithi053 ceph-mon[27968]: pgmap v258: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:18:06.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:06 smithi155 ceph-mon[35292]: pgmap v258: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:18:08.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:08 smithi053 ceph-mon[27968]: pgmap v259: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:18:08.727 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:08 smithi155 ceph-mon[35292]: pgmap v259: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:18:10.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:10 smithi053 ceph-mon[27968]: pgmap v260: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:18:10.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:10 smithi155 ceph-mon[35292]: pgmap v260: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:18:12.420 INFO:tasks.workunit.client.1.smithi155.stderr:Note: switching to '6fad5dd73c93545e52f57f681e140a9c3529c11e'. 2024-09-19T06:18:12.421 INFO:tasks.workunit.client.1.smithi155.stderr: 2024-09-19T06:18:12.421 INFO:tasks.workunit.client.1.smithi155.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-19T06:18:12.421 INFO:tasks.workunit.client.1.smithi155.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-19T06:18:12.421 INFO:tasks.workunit.client.1.smithi155.stderr:state without impacting any branches by switching back to a branch. 2024-09-19T06:18:12.421 INFO:tasks.workunit.client.1.smithi155.stderr: 2024-09-19T06:18:12.421 INFO:tasks.workunit.client.1.smithi155.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-19T06:18:12.421 INFO:tasks.workunit.client.1.smithi155.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-19T06:18:12.421 INFO:tasks.workunit.client.1.smithi155.stderr: 2024-09-19T06:18:12.421 INFO:tasks.workunit.client.1.smithi155.stderr: git switch -c 2024-09-19T06:18:12.421 INFO:tasks.workunit.client.1.smithi155.stderr: 2024-09-19T06:18:12.421 INFO:tasks.workunit.client.1.smithi155.stderr:Or undo this operation with: 2024-09-19T06:18:12.421 INFO:tasks.workunit.client.1.smithi155.stderr: 2024-09-19T06:18:12.422 INFO:tasks.workunit.client.1.smithi155.stderr: git switch - 2024-09-19T06:18:12.422 INFO:tasks.workunit.client.1.smithi155.stderr: 2024-09-19T06:18:12.422 INFO:tasks.workunit.client.1.smithi155.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-19T06:18:12.422 INFO:tasks.workunit.client.1.smithi155.stderr: 2024-09-19T06:18:12.422 INFO:tasks.workunit.client.1.smithi155.stderr:HEAD is now at 6fad5dd73c9 Merge pull request #59812 from neha-ojha/wip-rtd-squid 2024-09-19T06:18:12.426 DEBUG:teuthology.orchestra.run.smithi155:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-09-19T06:18:12.521 INFO:tasks.workunit.client.1.smithi155.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-19T06:18:12.523 INFO:tasks.workunit.client.1.smithi155.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-09-19T06:18:12.523 INFO:tasks.workunit.client.1.smithi155.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-19T06:18:12.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:12 smithi053 ceph-mon[27968]: pgmap v261: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:18:12.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:12 smithi155 ceph-mon[35292]: pgmap v261: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:18:13.217 INFO:tasks.workunit.client.1.smithi155.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-19T06:18:13.291 INFO:tasks.workunit.client.1.smithi155.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-19T06:18:13.331 INFO:tasks.workunit.client.1.smithi155.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-09-19T06:18:13.333 INFO:tasks.workunit.client.1.smithi155.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-09-19T06:18:13.333 INFO:tasks.workunit.client.1.smithi155.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-19T06:18:13.372 INFO:tasks.workunit.client.1.smithi155.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-09-19T06:18:13.376 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-19T06:18:13.376 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-09-19T06:18:13.393 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.1... 2024-09-19T06:18:13.394 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-09-19T06:18:13.394 DEBUG:teuthology.orchestra.run.smithi155:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/dbench.sh 2024-09-19T06:18:13.461 INFO:tasks.workunit.client.1.smithi155.stderr:+ dbench 1 2024-09-19T06:18:13.506 INFO:tasks.workunit.client.1.smithi155.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-19T06:18:13.507 INFO:tasks.workunit.client.1.smithi155.stdout: 2024-09-19T06:18:13.507 INFO:tasks.workunit.client.1.smithi155.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-19T06:18:13.516 INFO:tasks.workunit.client.1.smithi155.stdout:0 of 1 processes prepared for launch 0 sec 2024-09-19T06:18:13.616 INFO:tasks.workunit.client.1.smithi155.stdout:1 of 1 processes prepared for launch 0 sec 2024-09-19T06:18:13.616 INFO:tasks.workunit.client.1.smithi155.stdout:releasing clients 2024-09-19T06:18:14.554 INFO:tasks.workunit.client.0.smithi053.stderr:Updating files: 84% (10928/12896) Updating files: 85% (10962/12896) Updating files: 86% (11091/12896) Updating files: 87% (11220/12896) Updating files: 88% (11349/12896) Updating files: 89% (11478/12896) Updating files: 90% (11607/12896) Updating files: 91% (11736/12896) Updating files: 92% (11865/12896) Updating files: 93% (11994/12896) Updating files: 94% (12123/12896) Updating files: 95% (12252/12896) Updating files: 96% (12381/12896) Updating files: 97% (12510/12896) Updating files: 98% (12639/12896) Updating files: 99% (12768/12896) Updating files: 100% (12896/12896) Updating files: 100% (12896/12896), done. 2024-09-19T06:18:14.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:14 smithi053 ceph-mon[27968]: pgmap v262: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:18:14.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:14 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:18:14.616 INFO:tasks.workunit.client.1.smithi155.stdout: 1 980 21.90 MB/sec warmup 1 sec latency 13.695 ms 2024-09-19T06:18:14.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:14 smithi155 ceph-mon[35292]: pgmap v262: 129 pgs: 129 active+clean; 587 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:18:14.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:14 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:18:15.490 INFO:tasks.workunit.client.0.smithi053.stderr:Note: switching to '6fad5dd73c93545e52f57f681e140a9c3529c11e'. 2024-09-19T06:18:15.491 INFO:tasks.workunit.client.0.smithi053.stderr: 2024-09-19T06:18:15.491 INFO:tasks.workunit.client.0.smithi053.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-19T06:18:15.491 INFO:tasks.workunit.client.0.smithi053.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-19T06:18:15.491 INFO:tasks.workunit.client.0.smithi053.stderr:state without impacting any branches by switching back to a branch. 2024-09-19T06:18:15.491 INFO:tasks.workunit.client.0.smithi053.stderr: 2024-09-19T06:18:15.491 INFO:tasks.workunit.client.0.smithi053.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-19T06:18:15.491 INFO:tasks.workunit.client.0.smithi053.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-19T06:18:15.491 INFO:tasks.workunit.client.0.smithi053.stderr: 2024-09-19T06:18:15.491 INFO:tasks.workunit.client.0.smithi053.stderr: git switch -c 2024-09-19T06:18:15.491 INFO:tasks.workunit.client.0.smithi053.stderr: 2024-09-19T06:18:15.491 INFO:tasks.workunit.client.0.smithi053.stderr:Or undo this operation with: 2024-09-19T06:18:15.491 INFO:tasks.workunit.client.0.smithi053.stderr: 2024-09-19T06:18:15.491 INFO:tasks.workunit.client.0.smithi053.stderr: git switch - 2024-09-19T06:18:15.492 INFO:tasks.workunit.client.0.smithi053.stderr: 2024-09-19T06:18:15.492 INFO:tasks.workunit.client.0.smithi053.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-19T06:18:15.492 INFO:tasks.workunit.client.0.smithi053.stderr: 2024-09-19T06:18:15.492 INFO:tasks.workunit.client.0.smithi053.stderr:HEAD is now at 6fad5dd73c9 Merge pull request #59812 from neha-ojha/wip-rtd-squid 2024-09-19T06:18:15.496 DEBUG:teuthology.orchestra.run.smithi053:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-19T06:18:15.567 INFO:tasks.workunit.client.0.smithi053.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-19T06:18:15.570 INFO:tasks.workunit.client.0.smithi053.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T06:18:15.570 INFO:tasks.workunit.client.0.smithi053.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-19T06:18:15.616 INFO:tasks.workunit.client.1.smithi155.stdout: 1 1836 13.18 MB/sec warmup 2 sec latency 12.489 ms 2024-09-19T06:18:16.263 INFO:tasks.workunit.client.0.smithi053.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-19T06:18:16.355 INFO:tasks.workunit.client.0.smithi053.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-19T06:18:16.401 INFO:tasks.workunit.client.0.smithi053.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T06:18:16.403 INFO:tasks.workunit.client.0.smithi053.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T06:18:16.403 INFO:tasks.workunit.client.0.smithi053.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-19T06:18:16.446 INFO:tasks.workunit.client.0.smithi053.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T06:18:16.450 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-19T06:18:16.450 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-19T06:18:16.506 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-09-19T06:18:16.508 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-09-19T06:18:16.508 DEBUG:teuthology.orchestra.run.smithi053:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-09-19T06:18:16.567 INFO:tasks.workunit.client.0.smithi053.stderr:+ dbench 1 2024-09-19T06:18:16.577 INFO:tasks.workunit.client.0.smithi053.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-19T06:18:16.577 INFO:tasks.workunit.client.0.smithi053.stdout: 2024-09-19T06:18:16.577 INFO:tasks.workunit.client.0.smithi053.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-19T06:18:16.584 INFO:tasks.workunit.client.0.smithi053.stdout:0 of 1 processes prepared for launch 0 sec 2024-09-19T06:18:16.616 INFO:tasks.workunit.client.1.smithi155.stdout: 1 2607 9.67 MB/sec warmup 3 sec latency 11.258 ms 2024-09-19T06:18:16.684 INFO:tasks.workunit.client.0.smithi053.stdout:1 of 1 processes prepared for launch 0 sec 2024-09-19T06:18:16.684 INFO:tasks.workunit.client.0.smithi053.stdout:releasing clients 2024-09-19T06:18:16.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:16 smithi155 ceph-mon[35292]: pgmap v263: 129 pgs: 129 active+clean; 3.5 MiB data, 248 MiB used, 715 GiB / 715 GiB avail; 251 KiB/s wr, 2 op/s 2024-09-19T06:18:16.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:16 smithi053 ceph-mon[27968]: pgmap v263: 129 pgs: 129 active+clean; 3.5 MiB data, 248 MiB used, 715 GiB / 715 GiB avail; 251 KiB/s wr, 2 op/s 2024-09-19T06:18:17.616 INFO:tasks.workunit.client.1.smithi155.stdout: 1 3493 9.59 MB/sec warmup 4 sec latency 11.815 ms 2024-09-19T06:18:17.684 INFO:tasks.workunit.client.0.smithi053.stdout: 1 2477 28.72 MB/sec warmup 1 sec latency 184.753 ms 2024-09-19T06:18:18.616 INFO:tasks.workunit.client.1.smithi155.stdout: 1 4067 8.02 MB/sec warmup 5 sec latency 12.131 ms 2024-09-19T06:18:18.684 INFO:tasks.workunit.client.0.smithi053.stdout: 1 5199 22.65 MB/sec warmup 2 sec latency 44.887 ms 2024-09-19T06:18:18.727 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:18 smithi155 ceph-mon[35292]: pgmap v264: 129 pgs: 129 active+clean; 6.7 MiB data, 264 MiB used, 715 GiB / 715 GiB avail; 553 KiB/s wr, 5 op/s 2024-09-19T06:18:18.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:18 smithi053 ceph-mon[27968]: pgmap v264: 129 pgs: 129 active+clean; 6.7 MiB data, 264 MiB used, 715 GiB / 715 GiB avail; 553 KiB/s wr, 5 op/s 2024-09-19T06:18:19.616 INFO:tasks.workunit.client.1.smithi155.stdout: 1 4821 7.03 MB/sec warmup 6 sec latency 14.930 ms 2024-09-19T06:18:19.684 INFO:tasks.workunit.client.0.smithi053.stdout: 1 8025 20.00 MB/sec warmup 3 sec latency 42.363 ms 2024-09-19T06:18:20.616 INFO:tasks.workunit.client.1.smithi155.stdout: 1 5710 6.54 MB/sec warmup 7 sec latency 13.045 ms 2024-09-19T06:18:20.684 INFO:tasks.workunit.client.0.smithi053.stdout: 1 11456 19.71 MB/sec warmup 4 sec latency 34.439 ms 2024-09-19T06:18:20.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:20 smithi155 ceph-mon[35292]: pgmap v265: 129 pgs: 129 active+clean; 24 MiB data, 316 MiB used, 715 GiB / 715 GiB avail; 341 B/s rd, 2.2 MiB/s wr, 32 op/s 2024-09-19T06:18:20.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:20 smithi053 ceph-mon[27968]: pgmap v265: 129 pgs: 129 active+clean; 24 MiB data, 316 MiB used, 715 GiB / 715 GiB avail; 341 B/s rd, 2.2 MiB/s wr, 32 op/s 2024-09-19T06:18:21.616 INFO:tasks.workunit.client.1.smithi155.stdout: 1 6549 6.58 MB/sec warmup 8 sec latency 38.661 ms 2024-09-19T06:18:21.684 INFO:tasks.workunit.client.0.smithi053.stdout: 1 12289 16.71 MB/sec warmup 5 sec latency 274.393 ms 2024-09-19T06:18:22.616 INFO:tasks.workunit.client.1.smithi155.stdout: 1 7190 6.42 MB/sec warmup 9 sec latency 88.369 ms 2024-09-19T06:18:22.647 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:22 smithi053 ceph-mon[27968]: pgmap v266: 129 pgs: 129 active+clean; 24 MiB data, 316 MiB used, 715 GiB / 715 GiB avail; 341 B/s rd, 2.2 MiB/s wr, 34 op/s 2024-09-19T06:18:22.684 INFO:tasks.workunit.client.0.smithi053.stdout: 1 12546 13.96 MB/sec warmup 6 sec latency 357.334 ms 2024-09-19T06:18:22.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:22 smithi155 ceph-mon[35292]: pgmap v266: 129 pgs: 129 active+clean; 24 MiB data, 316 MiB used, 715 GiB / 715 GiB avail; 341 B/s rd, 2.2 MiB/s wr, 34 op/s 2024-09-19T06:18:23.616 INFO:tasks.workunit.client.1.smithi155.stdout: 1 7758 5.93 MB/sec warmup 10 sec latency 14.277 ms 2024-09-19T06:18:23.684 INFO:tasks.workunit.client.0.smithi053.stdout: 1 15188 14.06 MB/sec warmup 7 sec latency 175.222 ms 2024-09-19T06:18:24.616 INFO:tasks.workunit.client.1.smithi155.stdout: 1 8594 5.84 MB/sec warmup 11 sec latency 12.005 ms 2024-09-19T06:18:24.684 INFO:tasks.workunit.client.0.smithi053.stdout: 1 18667 14.67 MB/sec warmup 8 sec latency 107.992 ms 2024-09-19T06:18:24.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:24 smithi155 ceph-mon[35292]: pgmap v267: 129 pgs: 129 active+clean; 24 MiB data, 318 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.5 MiB/s wr, 46 op/s 2024-09-19T06:18:24.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:24 smithi053 ceph-mon[27968]: pgmap v267: 129 pgs: 129 active+clean; 24 MiB data, 318 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.5 MiB/s wr, 46 op/s 2024-09-19T06:18:25.616 INFO:tasks.workunit.client.1.smithi155.stdout: 1 9466 5.51 MB/sec warmup 12 sec latency 12.894 ms 2024-09-19T06:18:25.684 INFO:tasks.workunit.client.0.smithi053.stdout: 1 22211 15.16 MB/sec warmup 9 sec latency 141.391 ms 2024-09-19T06:18:26.616 INFO:tasks.workunit.client.1.smithi155.stdout: 1 10405 5.66 MB/sec warmup 13 sec latency 17.481 ms 2024-09-19T06:18:26.684 INFO:tasks.workunit.client.0.smithi053.stdout: 1 25798 15.57 MB/sec warmup 10 sec latency 41.023 ms 2024-09-19T06:18:26.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:26 smithi155 ceph-mon[35292]: pgmap v268: 129 pgs: 129 active+clean; 26 MiB data, 327 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.0 MiB/s wr, 73 op/s 2024-09-19T06:18:26.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:26 smithi053 ceph-mon[27968]: pgmap v268: 129 pgs: 129 active+clean; 26 MiB data, 327 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.0 MiB/s wr, 73 op/s 2024-09-19T06:18:27.616 INFO:tasks.workunit.client.1.smithi155.stdout: 1 11028 5.59 MB/sec warmup 14 sec latency 13.121 ms 2024-09-19T06:18:27.684 INFO:tasks.workunit.client.0.smithi053.stdout: 1 29224 15.86 MB/sec warmup 11 sec latency 74.486 ms 2024-09-19T06:18:28.616 INFO:tasks.workunit.client.1.smithi155.stdout: 1 11686 5.35 MB/sec warmup 15 sec latency 16.421 ms 2024-09-19T06:18:28.684 INFO:tasks.workunit.client.0.smithi053.stdout: 1 32709 16.12 MB/sec warmup 12 sec latency 33.770 ms 2024-09-19T06:18:28.727 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:28 smithi155 ceph-mon[35292]: pgmap v269: 129 pgs: 129 active+clean; 25 MiB data, 326 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.9 MiB/s wr, 77 op/s 2024-09-19T06:18:28.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:28 smithi053 ceph-mon[27968]: pgmap v269: 129 pgs: 129 active+clean; 25 MiB data, 326 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.9 MiB/s wr, 77 op/s 2024-09-19T06:18:29.616 INFO:tasks.workunit.client.1.smithi155.stdout: 1 12518 5.23 MB/sec warmup 16 sec latency 13.826 ms 2024-09-19T06:18:29.684 INFO:tasks.workunit.client.0.smithi053.stdout: 1 36241 16.35 MB/sec warmup 13 sec latency 44.017 ms 2024-09-19T06:18:29.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:29 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:18:29.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:18:30.616 INFO:tasks.workunit.client.1.smithi155.stdout: 1 13370 5.25 MB/sec warmup 17 sec latency 16.662 ms 2024-09-19T06:18:30.685 INFO:tasks.workunit.client.0.smithi053.stdout: 1 39741 16.54 MB/sec warmup 14 sec latency 32.784 ms 2024-09-19T06:18:30.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:30 smithi155 ceph-mon[35292]: pgmap v270: 129 pgs: 129 active+clean; 27 MiB data, 335 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.2 MiB/s wr, 104 op/s 2024-09-19T06:18:30.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:30 smithi053 ceph-mon[27968]: pgmap v270: 129 pgs: 129 active+clean; 27 MiB data, 335 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.2 MiB/s wr, 104 op/s 2024-09-19T06:18:31.616 INFO:tasks.workunit.client.1.smithi155.stdout: 1 14218 5.32 MB/sec warmup 18 sec latency 17.616 ms 2024-09-19T06:18:31.685 INFO:tasks.workunit.client.0.smithi053.stdout: 1 43274 16.71 MB/sec warmup 15 sec latency 41.367 ms 2024-09-19T06:18:32.616 INFO:tasks.workunit.client.1.smithi155.stdout: 1 14801 5.13 MB/sec warmup 19 sec latency 9.720 ms 2024-09-19T06:18:32.647 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:32 smithi053 ceph-mon[27968]: pgmap v271: 129 pgs: 129 active+clean; 27 MiB data, 336 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.6 MiB/s wr, 79 op/s 2024-09-19T06:18:32.684 INFO:tasks.workunit.client.0.smithi053.stdout: 1 45925 16.55 MB/sec warmup 16 sec latency 41.746 ms 2024-09-19T06:18:32.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:32 smithi155 ceph-mon[35292]: pgmap v271: 129 pgs: 129 active+clean; 27 MiB data, 336 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.6 MiB/s wr, 79 op/s 2024-09-19T06:18:33.616 INFO:tasks.workunit.client.1.smithi155.stdout: 1 15584 5.12 MB/sec warmup 20 sec latency 19.581 ms 2024-09-19T06:18:33.685 INFO:tasks.workunit.client.0.smithi053.stdout: 1 49143 16.60 MB/sec warmup 17 sec latency 50.149 ms 2024-09-19T06:18:34.616 INFO:tasks.workunit.client.1.smithi155.stdout: 1 16380 4.92 MB/sec warmup 21 sec latency 13.487 ms 2024-09-19T06:18:34.685 INFO:tasks.workunit.client.0.smithi053.stdout: 1 52609 16.73 MB/sec warmup 18 sec latency 30.981 ms 2024-09-19T06:18:34.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:34 smithi155 ceph-mon[35292]: pgmap v272: 129 pgs: 129 active+clean; 29 MiB data, 343 MiB used, 715 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.1 MiB/s wr, 106 op/s 2024-09-19T06:18:34.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:34 smithi053 ceph-mon[27968]: pgmap v272: 129 pgs: 129 active+clean; 29 MiB data, 343 MiB used, 715 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.1 MiB/s wr, 106 op/s 2024-09-19T06:18:35.616 INFO:tasks.workunit.client.1.smithi155.stdout: 1 17438 5.06 MB/sec warmup 22 sec latency 16.051 ms 2024-09-19T06:18:35.685 INFO:tasks.workunit.client.0.smithi053.stdout: 1 55614 16.57 MB/sec warmup 19 sec latency 31.913 ms 2024-09-19T06:18:36.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:36 smithi053 ceph-mon[27968]: pgmap v273: 129 pgs: 129 active+clean; 29 MiB data, 344 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.1 MiB/s wr, 105 op/s 2024-09-19T06:18:36.616 INFO:tasks.workunit.client.1.smithi155.stdout: 1 18031 5.05 MB/sec warmup 23 sec latency 11.181 ms 2024-09-19T06:18:36.685 INFO:tasks.workunit.client.0.smithi053.stdout: 1 58670 16.62 MB/sec warmup 20 sec latency 43.395 ms 2024-09-19T06:18:36.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:36 smithi155 ceph-mon[35292]: pgmap v273: 129 pgs: 129 active+clean; 29 MiB data, 344 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.1 MiB/s wr, 105 op/s 2024-09-19T06:18:37.616 INFO:tasks.workunit.client.1.smithi155.stdout: 1 18636 4.89 MB/sec warmup 24 sec latency 14.851 ms 2024-09-19T06:18:37.685 INFO:tasks.workunit.client.0.smithi053.stdout: 1 61318 16.52 MB/sec warmup 21 sec latency 34.467 ms 2024-09-19T06:18:38.616 INFO:tasks.workunit.client.1.smithi155.stdout: 1 19457 4.87 MB/sec warmup 25 sec latency 12.177 ms 2024-09-19T06:18:38.685 INFO:tasks.workunit.client.0.smithi053.stdout: 1 64753 16.63 MB/sec warmup 22 sec latency 36.097 ms 2024-09-19T06:18:38.727 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:38 smithi155 ceph-mon[35292]: pgmap v274: 129 pgs: 129 active+clean; 28 MiB data, 344 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 MiB/s wr, 84 op/s 2024-09-19T06:18:38.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:38 smithi053 ceph-mon[27968]: pgmap v274: 129 pgs: 129 active+clean; 28 MiB data, 344 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 MiB/s wr, 84 op/s 2024-09-19T06:18:39.616 INFO:tasks.workunit.client.1.smithi155.stdout: 1 20334 4.79 MB/sec warmup 26 sec latency 15.550 ms 2024-09-19T06:18:39.685 INFO:tasks.workunit.client.0.smithi053.stdout: 1 68233 16.73 MB/sec warmup 23 sec latency 33.704 ms 2024-09-19T06:18:40.616 INFO:tasks.workunit.client.1.smithi155.stdout: 1 21139 4.86 MB/sec warmup 27 sec latency 11.702 ms 2024-09-19T06:18:40.685 INFO:tasks.workunit.client.0.smithi053.stdout: 1 71768 16.83 MB/sec warmup 24 sec latency 40.581 ms 2024-09-19T06:18:40.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:40 smithi155 ceph-mon[35292]: pgmap v275: 129 pgs: 129 active+clean; 30 MiB data, 360 MiB used, 715 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.2 MiB/s wr, 109 op/s 2024-09-19T06:18:40.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:40 smithi053 ceph-mon[27968]: pgmap v275: 129 pgs: 129 active+clean; 30 MiB data, 360 MiB used, 715 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.2 MiB/s wr, 109 op/s 2024-09-19T06:18:41.616 INFO:tasks.workunit.client.1.smithi155.stdout: 1 21828 4.85 MB/sec warmup 28 sec latency 11.898 ms 2024-09-19T06:18:41.685 INFO:tasks.workunit.client.0.smithi053.stdout: 1 75283 16.92 MB/sec warmup 25 sec latency 100.307 ms 2024-09-19T06:18:42.616 INFO:tasks.workunit.client.1.smithi155.stdout: 1 22556 4.75 MB/sec warmup 29 sec latency 17.253 ms 2024-09-19T06:18:42.647 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:42 smithi053 ceph-mon[27968]: pgmap v276: 129 pgs: 129 active+clean; 30 MiB data, 364 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 81 op/s 2024-09-19T06:18:42.685 INFO:tasks.workunit.client.0.smithi053.stdout: 1 78819 17.00 MB/sec warmup 26 sec latency 44.031 ms 2024-09-19T06:18:42.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:42 smithi155 ceph-mon[35292]: pgmap v276: 129 pgs: 129 active+clean; 30 MiB data, 364 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 81 op/s 2024-09-19T06:18:43.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 23348 4.71 MB/sec warmup 30 sec latency 53.290 ms 2024-09-19T06:18:43.685 INFO:tasks.workunit.client.0.smithi053.stdout: 1 81083 16.83 MB/sec warmup 27 sec latency 278.828 ms 2024-09-19T06:18:44.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 24106 4.73 MB/sec warmup 31 sec latency 15.578 ms 2024-09-19T06:18:44.685 INFO:tasks.workunit.client.0.smithi053.stdout: 1 81412 16.29 MB/sec warmup 28 sec latency 286.006 ms 2024-09-19T06:18:44.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:44 smithi155 ceph-mon[35292]: pgmap v277: 129 pgs: 129 active+clean; 32 MiB data, 366 MiB used, 715 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.0 MiB/s wr, 109 op/s 2024-09-19T06:18:44.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:44 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:18:44.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:44 smithi053 ceph-mon[27968]: pgmap v277: 129 pgs: 129 active+clean; 32 MiB data, 366 MiB used, 715 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.0 MiB/s wr, 109 op/s 2024-09-19T06:18:44.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:44 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:18:45.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 24881 4.79 MB/sec warmup 32 sec latency 18.841 ms 2024-09-19T06:18:45.685 INFO:tasks.workunit.client.0.smithi053.stdout: 1 82321 15.90 MB/sec warmup 29 sec latency 249.146 ms 2024-09-19T06:18:46.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 25386 4.69 MB/sec warmup 33 sec latency 88.528 ms 2024-09-19T06:18:46.685 INFO:tasks.workunit.client.0.smithi053.stdout: 1 82628 15.39 MB/sec warmup 30 sec latency 327.086 ms 2024-09-19T06:18:46.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:46 smithi155 ceph-mon[35292]: pgmap v278: 129 pgs: 129 active+clean; 32 MiB data, 373 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.8 MiB/s wr, 90 op/s 2024-09-19T06:18:46.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:46 smithi053 ceph-mon[27968]: pgmap v278: 129 pgs: 129 active+clean; 32 MiB data, 373 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.8 MiB/s wr, 90 op/s 2024-09-19T06:18:47.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 26085 4.62 MB/sec warmup 34 sec latency 20.890 ms 2024-09-19T06:18:47.685 INFO:tasks.workunit.client.0.smithi053.stdout: 1 85921 15.49 MB/sec warmup 31 sec latency 125.149 ms 2024-09-19T06:18:48.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 26897 4.58 MB/sec warmup 35 sec latency 18.167 ms 2024-09-19T06:18:48.685 INFO:tasks.workunit.client.0.smithi053.stdout: 1 89409 15.60 MB/sec warmup 32 sec latency 120.876 ms 2024-09-19T06:18:48.728 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:48 smithi155 ceph-mon[35292]: pgmap v279: 129 pgs: 129 active+clean; 32 MiB data, 372 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.7 MiB/s wr, 97 op/s 2024-09-19T06:18:48.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:48 smithi053 ceph-mon[27968]: pgmap v279: 129 pgs: 129 active+clean; 32 MiB data, 372 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.7 MiB/s wr, 97 op/s 2024-09-19T06:18:49.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 27788 4.65 MB/sec warmup 36 sec latency 17.193 ms 2024-09-19T06:18:49.686 INFO:tasks.workunit.client.0.smithi053.stdout: 1 89612 15.14 MB/sec warmup 33 sec latency 275.146 ms 2024-09-19T06:18:50.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 28423 4.66 MB/sec warmup 37 sec latency 47.093 ms 2024-09-19T06:18:50.685 INFO:tasks.workunit.client.0.smithi053.stdout: 1 89716 14.71 MB/sec warmup 34 sec latency 300.130 ms 2024-09-19T06:18:50.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:50 smithi155 ceph-mon[35292]: pgmap v280: 129 pgs: 129 active+clean; 33 MiB data, 377 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.1 MiB/s wr, 104 op/s 2024-09-19T06:18:50.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:50 smithi053 ceph-mon[27968]: pgmap v280: 129 pgs: 129 active+clean; 33 MiB data, 377 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.1 MiB/s wr, 104 op/s 2024-09-19T06:18:51.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 28932 4.58 MB/sec warmup 38 sec latency 37.428 ms 2024-09-19T06:18:51.685 INFO:tasks.workunit.client.0.smithi053.stdout: 1 90364 14.41 MB/sec warmup 35 sec latency 305.697 ms 2024-09-19T06:18:52.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 29599 4.51 MB/sec warmup 39 sec latency 43.859 ms 2024-09-19T06:18:52.647 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:52 smithi053 ceph-mon[27968]: pgmap v281: 129 pgs: 129 active+clean; 33 MiB data, 377 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.5 MiB/s wr, 75 op/s 2024-09-19T06:18:52.686 INFO:tasks.workunit.client.0.smithi053.stdout: 1 90620 14.01 MB/sec warmup 36 sec latency 290.567 ms 2024-09-19T06:18:52.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:52 smithi155 ceph-mon[35292]: pgmap v281: 129 pgs: 129 active+clean; 33 MiB data, 377 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.5 MiB/s wr, 75 op/s 2024-09-19T06:18:53.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 30343 4.49 MB/sec warmup 40 sec latency 51.211 ms 2024-09-19T06:18:53.685 INFO:tasks.workunit.client.0.smithi053.stdout: 1 91144 13.68 MB/sec warmup 37 sec latency 301.056 ms 2024-09-19T06:18:54.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 31175 4.51 MB/sec warmup 41 sec latency 45.040 ms 2024-09-19T06:18:54.686 INFO:tasks.workunit.client.0.smithi053.stdout: 1 91705 13.46 MB/sec warmup 38 sec latency 324.790 ms 2024-09-19T06:18:54.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:54 smithi155 ceph-mon[35292]: pgmap v282: 129 pgs: 129 active+clean; 34 MiB data, 388 MiB used, 715 GiB / 715 GiB avail; 5.1 KiB/s rd, 1.9 MiB/s wr, 100 op/s 2024-09-19T06:18:54.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:54 smithi053 ceph-mon[27968]: pgmap v282: 129 pgs: 129 active+clean; 34 MiB data, 388 MiB used, 715 GiB / 715 GiB avail; 5.1 KiB/s rd, 1.9 MiB/s wr, 100 op/s 2024-09-19T06:18:55.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 31963 4.56 MB/sec warmup 42 sec latency 45.083 ms 2024-09-19T06:18:55.686 INFO:tasks.workunit.client.0.smithi053.stdout: 1 92027 13.17 MB/sec warmup 39 sec latency 300.045 ms 2024-09-19T06:18:56.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 32553 4.49 MB/sec warmup 43 sec latency 26.070 ms 2024-09-19T06:18:56.657 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:56 smithi053 ceph-mon[27968]: pgmap v283: 129 pgs: 129 active+clean; 34 MiB data, 389 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.8 MiB/s wr, 80 op/s 2024-09-19T06:18:56.686 INFO:tasks.workunit.client.0.smithi053.stdout: 1 93007 12.96 MB/sec warmup 40 sec latency 291.728 ms 2024-09-19T06:18:56.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:56 smithi155 ceph-mon[35292]: pgmap v283: 129 pgs: 129 active+clean; 34 MiB data, 389 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.8 MiB/s wr, 80 op/s 2024-09-19T06:18:57.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 33374 4.50 MB/sec warmup 44 sec latency 15.665 ms 2024-09-19T06:18:57.686 INFO:tasks.workunit.client.0.smithi053.stdout: 1 96505 13.11 MB/sec warmup 41 sec latency 50.163 ms 2024-09-19T06:18:58.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 34273 4.44 MB/sec warmup 45 sec latency 12.527 ms 2024-09-19T06:18:58.686 INFO:tasks.workunit.client.0.smithi053.stdout: 1 100040 13.25 MB/sec warmup 42 sec latency 50.062 ms 2024-09-19T06:18:58.728 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:58 smithi155 ceph-mon[35292]: pgmap v284: 129 pgs: 129 active+clean; 35 MiB data, 399 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 MiB/s wr, 90 op/s 2024-09-19T06:18:58.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:58 smithi053 ceph-mon[27968]: pgmap v284: 129 pgs: 129 active+clean; 35 MiB data, 399 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 MiB/s wr, 90 op/s 2024-09-19T06:18:59.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 35243 4.51 MB/sec warmup 46 sec latency 14.324 ms 2024-09-19T06:18:59.686 INFO:tasks.workunit.client.0.smithi053.stdout: 1 102662 13.28 MB/sec warmup 43 sec latency 33.760 ms 2024-09-19T06:18:59.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:18:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:18:59.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:18:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:19:00.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 35787 4.51 MB/sec warmup 47 sec latency 15.728 ms 2024-09-19T06:19:00.686 INFO:tasks.workunit.client.0.smithi053.stdout: 1 105829 13.37 MB/sec warmup 44 sec latency 39.543 ms 2024-09-19T06:19:00.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:00 smithi155 ceph-mon[35292]: pgmap v285: 129 pgs: 129 active+clean; 35 MiB data, 388 MiB used, 715 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.0 MiB/s wr, 87 op/s 2024-09-19T06:19:00.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:00 smithi053 ceph-mon[27968]: pgmap v285: 129 pgs: 129 active+clean; 35 MiB data, 388 MiB used, 715 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.0 MiB/s wr, 87 op/s 2024-09-19T06:19:01.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 36372 4.43 MB/sec warmup 48 sec latency 14.287 ms 2024-09-19T06:19:01.686 INFO:tasks.workunit.client.0.smithi053.stdout: 1 108868 13.37 MB/sec warmup 45 sec latency 58.410 ms 2024-09-19T06:19:01.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:01 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:19:01.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:01 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:19:01.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:01 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:19:01.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:01 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:19:01.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:01 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:19:01.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:01 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:19:01.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:01 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:19:01.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:01 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:19:02.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 37221 4.44 MB/sec warmup 49 sec latency 9.673 ms 2024-09-19T06:19:02.686 INFO:tasks.workunit.client.0.smithi053.stdout: 1 111892 13.46 MB/sec warmup 46 sec latency 33.482 ms 2024-09-19T06:19:02.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:02 smithi155 ceph-mon[35292]: pgmap v286: 129 pgs: 129 active+clean; 35 MiB data, 388 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.5 MiB/s wr, 75 op/s 2024-09-19T06:19:02.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:02 smithi053 ceph-mon[27968]: pgmap v286: 129 pgs: 129 active+clean; 35 MiB data, 388 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.5 MiB/s wr, 75 op/s 2024-09-19T06:19:03.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 37982 4.40 MB/sec warmup 50 sec latency 12.387 ms 2024-09-19T06:19:03.686 INFO:tasks.workunit.client.0.smithi053.stdout: 1 114533 13.49 MB/sec warmup 47 sec latency 108.377 ms 2024-09-19T06:19:04.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 38836 4.44 MB/sec warmup 51 sec latency 12.478 ms 2024-09-19T06:19:04.686 INFO:tasks.workunit.client.0.smithi053.stdout: 1 118013 13.60 MB/sec warmup 48 sec latency 53.781 ms 2024-09-19T06:19:04.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:04 smithi155 ceph-mon[35292]: pgmap v287: 129 pgs: 129 active+clean; 37 MiB data, 390 MiB used, 715 GiB / 715 GiB avail; 5.1 KiB/s rd, 1.9 MiB/s wr, 108 op/s 2024-09-19T06:19:04.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:04 smithi053 ceph-mon[27968]: pgmap v287: 129 pgs: 129 active+clean; 37 MiB data, 390 MiB used, 715 GiB / 715 GiB avail; 5.1 KiB/s rd, 1.9 MiB/s wr, 108 op/s 2024-09-19T06:19:05.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 39508 4.45 MB/sec warmup 52 sec latency 11.233 ms 2024-09-19T06:19:05.686 INFO:tasks.workunit.client.0.smithi053.stdout: 1 121443 13.71 MB/sec warmup 49 sec latency 30.140 ms 2024-09-19T06:19:06.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 40230 4.40 MB/sec warmup 53 sec latency 16.066 ms 2024-09-19T06:19:06.686 INFO:tasks.workunit.client.0.smithi053.stdout: 1 124927 13.81 MB/sec warmup 50 sec latency 116.787 ms 2024-09-19T06:19:06.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:06 smithi155 ceph-mon[35292]: pgmap v288: 129 pgs: 129 active+clean; 37 MiB data, 394 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.9 MiB/s wr, 90 op/s 2024-09-19T06:19:06.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:06 smithi053 ceph-mon[27968]: pgmap v288: 129 pgs: 129 active+clean; 37 MiB data, 394 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.9 MiB/s wr, 90 op/s 2024-09-19T06:19:07.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 41121 4.39 MB/sec warmup 54 sec latency 13.069 ms 2024-09-19T06:19:07.686 INFO:tasks.workunit.client.0.smithi053.stdout: 1 128468 13.92 MB/sec warmup 51 sec latency 33.647 ms 2024-09-19T06:19:08.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 42032 4.43 MB/sec warmup 55 sec latency 18.827 ms 2024-09-19T06:19:08.686 INFO:tasks.workunit.client.0.smithi053.stdout: 1 131962 14.01 MB/sec warmup 52 sec latency 58.389 ms 2024-09-19T06:19:08.728 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:08 smithi155 ceph-mon[35292]: pgmap v289: 129 pgs: 129 active+clean; 37 MiB data, 396 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.7 MiB/s wr, 86 op/s 2024-09-19T06:19:08.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:08 smithi053 ceph-mon[27968]: pgmap v289: 129 pgs: 129 active+clean; 37 MiB data, 396 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.7 MiB/s wr, 86 op/s 2024-09-19T06:19:09.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 42830 4.47 MB/sec warmup 56 sec latency 12.193 ms 2024-09-19T06:19:09.686 INFO:tasks.workunit.client.0.smithi053.stdout: 1 135497 14.11 MB/sec warmup 53 sec latency 32.162 ms 2024-09-19T06:19:10.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 43421 4.40 MB/sec warmup 57 sec latency 15.834 ms 2024-09-19T06:19:10.686 INFO:tasks.workunit.client.0.smithi053.stdout: 1 138123 14.11 MB/sec warmup 54 sec latency 26.521 ms 2024-09-19T06:19:10.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:10 smithi155 ceph-mon[35292]: pgmap v290: 129 pgs: 129 active+clean; 38 MiB data, 419 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.0 MiB/s wr, 95 op/s 2024-09-19T06:19:10.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:10 smithi053 ceph-mon[27968]: pgmap v290: 129 pgs: 129 active+clean; 38 MiB data, 419 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.0 MiB/s wr, 95 op/s 2024-09-19T06:19:11.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 44209 4.41 MB/sec warmup 58 sec latency 13.301 ms 2024-09-19T06:19:11.686 INFO:tasks.workunit.client.0.smithi053.stdout: 1 141288 14.17 MB/sec warmup 55 sec latency 66.717 ms 2024-09-19T06:19:12.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:12 smithi053 ceph-mon[27968]: pgmap v291: 129 pgs: 129 active+clean; 37 MiB data, 419 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.6 MiB/s wr, 85 op/s 2024-09-19T06:19:12.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 45009 4.36 MB/sec warmup 59 sec latency 18.900 ms 2024-09-19T06:19:12.686 INFO:tasks.workunit.client.0.smithi053.stdout: 1 144257 14.16 MB/sec warmup 56 sec latency 40.337 ms 2024-09-19T06:19:12.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:12 smithi155 ceph-mon[35292]: pgmap v291: 129 pgs: 129 active+clean; 37 MiB data, 419 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.6 MiB/s wr, 85 op/s 2024-09-19T06:19:13.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 45919 4.41 MB/sec warmup 60 sec latency 12.569 ms 2024-09-19T06:19:13.686 INFO:tasks.workunit.client.0.smithi053.stdout: 1 147352 14.22 MB/sec warmup 57 sec latency 30.347 ms 2024-09-19T06:19:14.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 46586 4.42 MB/sec warmup 61 sec latency 13.320 ms 2024-09-19T06:19:14.686 INFO:tasks.workunit.client.0.smithi053.stdout: 1 150640 14.30 MB/sec warmup 58 sec latency 25.381 ms 2024-09-19T06:19:14.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:14 smithi155 ceph-mon[35292]: pgmap v292: 129 pgs: 129 active+clean; 38 MiB data, 423 MiB used, 715 GiB / 715 GiB avail; 5.5 KiB/s rd, 2.0 MiB/s wr, 113 op/s 2024-09-19T06:19:14.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:14 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:19:14.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:14 smithi053 ceph-mon[27968]: pgmap v292: 129 pgs: 129 active+clean; 38 MiB data, 423 MiB used, 715 GiB / 715 GiB avail; 5.5 KiB/s rd, 2.0 MiB/s wr, 113 op/s 2024-09-19T06:19:14.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:14 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:19:15.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 47302 4.38 MB/sec warmup 62 sec latency 17.593 ms 2024-09-19T06:19:15.686 INFO:tasks.workunit.client.0.smithi053.stdout: 1 153543 14.31 MB/sec warmup 59 sec latency 58.134 ms 2024-09-19T06:19:16.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 48199 4.37 MB/sec warmup 63 sec latency 11.000 ms 2024-09-19T06:19:16.686 INFO:tasks.workunit.client.0.smithi053.stdout: 1 157085 14.39 MB/sec warmup 60 sec latency 27.515 ms 2024-09-19T06:19:16.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:16 smithi155 ceph-mon[35292]: pgmap v293: 129 pgs: 129 active+clean; 39 MiB data, 423 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 2.0 MiB/s wr, 87 op/s 2024-09-19T06:19:16.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:16 smithi053 ceph-mon[27968]: pgmap v293: 129 pgs: 129 active+clean; 39 MiB data, 423 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 2.0 MiB/s wr, 87 op/s 2024-09-19T06:19:17.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 49083 4.41 MB/sec warmup 64 sec latency 13.406 ms 2024-09-19T06:19:17.686 INFO:tasks.workunit.client.0.smithi053.stdout: 1 160563 14.46 MB/sec warmup 61 sec latency 41.851 ms 2024-09-19T06:19:18.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 49840 4.42 MB/sec warmup 65 sec latency 14.366 ms 2024-09-19T06:19:18.686 INFO:tasks.workunit.client.0.smithi053.stdout: 1 163925 14.53 MB/sec warmup 62 sec latency 133.391 ms 2024-09-19T06:19:18.729 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:18 smithi155 ceph-mon[35292]: pgmap v294: 129 pgs: 129 active+clean; 39 MiB data, 430 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.7 MiB/s wr, 84 op/s 2024-09-19T06:19:18.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:18 smithi053 ceph-mon[27968]: pgmap v294: 129 pgs: 129 active+clean; 39 MiB data, 430 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.7 MiB/s wr, 84 op/s 2024-09-19T06:19:19.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 50407 4.38 MB/sec warmup 66 sec latency 19.914 ms 2024-09-19T06:19:19.687 INFO:tasks.workunit.client.0.smithi053.stdout: 1 167467 14.60 MB/sec warmup 63 sec latency 49.730 ms 2024-09-19T06:19:20.617 INFO:tasks.workunit.client.1.smithi155.stdout: 1 51196 4.38 MB/sec warmup 67 sec latency 12.669 ms 2024-09-19T06:19:20.687 INFO:tasks.workunit.client.0.smithi053.stdout: 1 170958 14.67 MB/sec warmup 64 sec latency 26.793 ms 2024-09-19T06:19:20.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:20 smithi155 ceph-mon[35292]: pgmap v295: 129 pgs: 129 active+clean; 41 MiB data, 439 MiB used, 715 GiB / 715 GiB avail; 5.4 KiB/s rd, 2.2 MiB/s wr, 110 op/s 2024-09-19T06:19:20.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:20 smithi053 ceph-mon[27968]: pgmap v295: 129 pgs: 129 active+clean; 41 MiB data, 439 MiB used, 715 GiB / 715 GiB avail; 5.4 KiB/s rd, 2.2 MiB/s wr, 110 op/s 2024-09-19T06:19:21.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 52058 4.35 MB/sec warmup 68 sec latency 11.323 ms 2024-09-19T06:19:21.687 INFO:tasks.workunit.client.0.smithi053.stdout: 1 174501 14.74 MB/sec warmup 65 sec latency 33.518 ms 2024-09-19T06:19:22.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 52980 4.39 MB/sec warmup 69 sec latency 29.938 ms 2024-09-19T06:19:22.687 INFO:tasks.workunit.client.0.smithi053.stdout: 1 176168 14.62 MB/sec warmup 66 sec latency 283.590 ms 2024-09-19T06:19:22.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:22 smithi155 ceph-mon[35292]: pgmap v296: 129 pgs: 129 active+clean; 41 MiB data, 439 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.7 MiB/s wr, 81 op/s 2024-09-19T06:19:22.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:22 smithi053 ceph-mon[27968]: pgmap v296: 129 pgs: 129 active+clean; 41 MiB data, 439 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.7 MiB/s wr, 81 op/s 2024-09-19T06:19:23.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 53600 4.40 MB/sec warmup 70 sec latency 9.300 ms 2024-09-19T06:19:23.687 INFO:tasks.workunit.client.0.smithi053.stdout: 1 178289 14.60 MB/sec warmup 67 sec latency 116.829 ms 2024-09-19T06:19:24.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 54180 4.35 MB/sec warmup 71 sec latency 17.306 ms 2024-09-19T06:19:24.687 INFO:tasks.workunit.client.0.smithi053.stdout: 1 181713 14.66 MB/sec warmup 68 sec latency 75.197 ms 2024-09-19T06:19:24.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:24 smithi155 ceph-mon[35292]: pgmap v297: 129 pgs: 129 active+clean; 42 MiB data, 430 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 2.0 MiB/s wr, 109 op/s 2024-09-19T06:19:24.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:24 smithi053 ceph-mon[27968]: pgmap v297: 129 pgs: 129 active+clean; 42 MiB data, 430 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 2.0 MiB/s wr, 109 op/s 2024-09-19T06:19:25.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 55048 4.35 MB/sec warmup 72 sec latency 11.737 ms 2024-09-19T06:19:25.687 INFO:tasks.workunit.client.0.smithi053.stdout: 1 185193 14.72 MB/sec warmup 69 sec latency 33.586 ms 2024-09-19T06:19:26.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 55795 4.33 MB/sec warmup 73 sec latency 19.966 ms 2024-09-19T06:19:26.687 INFO:tasks.workunit.client.0.smithi053.stdout: 1 188691 14.78 MB/sec warmup 70 sec latency 41.458 ms 2024-09-19T06:19:26.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:26 smithi155 ceph-mon[35292]: pgmap v298: 129 pgs: 129 active+clean; 42 MiB data, 440 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.0 MiB/s wr, 88 op/s 2024-09-19T06:19:26.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:26 smithi053 ceph-mon[27968]: pgmap v298: 129 pgs: 129 active+clean; 42 MiB data, 440 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.0 MiB/s wr, 88 op/s 2024-09-19T06:19:27.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 56624 4.36 MB/sec warmup 74 sec latency 11.628 ms 2024-09-19T06:19:27.687 INFO:tasks.workunit.client.0.smithi053.stdout: 1 191307 14.77 MB/sec warmup 71 sec latency 108.445 ms 2024-09-19T06:19:28.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 57252 4.36 MB/sec warmup 75 sec latency 80.080 ms 2024-09-19T06:19:28.687 INFO:tasks.workunit.client.0.smithi053.stdout: 1 192288 14.64 MB/sec warmup 72 sec latency 283.496 ms 2024-09-19T06:19:28.729 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:28 smithi155 ceph-mon[35292]: pgmap v299: 129 pgs: 129 active+clean; 42 MiB data, 424 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.8 MiB/s wr, 87 op/s 2024-09-19T06:19:28.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:28 smithi053 ceph-mon[27968]: pgmap v299: 129 pgs: 129 active+clean; 42 MiB data, 424 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.8 MiB/s wr, 87 op/s 2024-09-19T06:19:29.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 57935 4.33 MB/sec warmup 76 sec latency 32.073 ms 2024-09-19T06:19:29.687 INFO:tasks.workunit.client.0.smithi053.stdout: 1 192349 14.44 MB/sec warmup 73 sec latency 308.446 ms 2024-09-19T06:19:29.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:29 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:19:29.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:19:30.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 58759 4.32 MB/sec warmup 77 sec latency 42.187 ms 2024-09-19T06:19:30.687 INFO:tasks.workunit.client.0.smithi053.stdout: 1 192359 14.24 MB/sec warmup 74 sec latency 300.247 ms 2024-09-19T06:19:30.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:30 smithi155 ceph-mon[35292]: pgmap v300: 129 pgs: 129 active+clean; 43 MiB data, 408 MiB used, 715 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.1 MiB/s wr, 108 op/s 2024-09-19T06:19:30.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:30 smithi053 ceph-mon[27968]: pgmap v300: 129 pgs: 129 active+clean; 43 MiB data, 408 MiB used, 715 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.1 MiB/s wr, 108 op/s 2024-09-19T06:19:31.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 59538 4.33 MB/sec warmup 78 sec latency 51.756 ms 2024-09-19T06:19:31.687 INFO:tasks.workunit.client.0.smithi053.stdout: 1 192470 14.06 MB/sec warmup 75 sec latency 308.441 ms 2024-09-19T06:19:32.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 60306 4.36 MB/sec warmup 79 sec latency 47.282 ms 2024-09-19T06:19:32.687 INFO:tasks.workunit.client.0.smithi053.stdout: 1 193035 13.93 MB/sec warmup 76 sec latency 316.853 ms 2024-09-19T06:19:32.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:32 smithi155 ceph-mon[35292]: pgmap v301: 129 pgs: 129 active+clean; 43 MiB data, 409 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.5 MiB/s wr, 79 op/s 2024-09-19T06:19:32.758 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:32 smithi053 ceph-mon[27968]: pgmap v301: 129 pgs: 129 active+clean; 43 MiB data, 409 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.5 MiB/s wr, 79 op/s 2024-09-19T06:19:33.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 60863 4.33 MB/sec warmup 80 sec latency 35.564 ms 2024-09-19T06:19:33.687 INFO:tasks.workunit.client.0.smithi053.stdout: 1 193439 13.75 MB/sec warmup 77 sec latency 307.456 ms 2024-09-19T06:19:34.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 61540 4.30 MB/sec warmup 81 sec latency 50.585 ms 2024-09-19T06:19:34.687 INFO:tasks.workunit.client.0.smithi053.stdout: 1 193925 13.60 MB/sec warmup 78 sec latency 308.576 ms 2024-09-19T06:19:34.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:34 smithi155 ceph-mon[35292]: pgmap v302: 129 pgs: 129 active+clean; 44 MiB data, 412 MiB used, 715 GiB / 715 GiB avail; 4.9 KiB/s rd, 1.9 MiB/s wr, 102 op/s 2024-09-19T06:19:34.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:34 smithi053 ceph-mon[27968]: pgmap v302: 129 pgs: 129 active+clean; 44 MiB data, 412 MiB used, 715 GiB / 715 GiB avail; 4.9 KiB/s rd, 1.9 MiB/s wr, 102 op/s 2024-09-19T06:19:35.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 62393 4.29 MB/sec warmup 82 sec latency 56.490 ms 2024-09-19T06:19:35.687 INFO:tasks.workunit.client.0.smithi053.stdout: 1 194467 13.50 MB/sec warmup 79 sec latency 315.259 ms 2024-09-19T06:19:36.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 63297 4.32 MB/sec warmup 83 sec latency 45.329 ms 2024-09-19T06:19:36.687 INFO:tasks.workunit.client.0.smithi053.stdout: 1 194742 13.34 MB/sec warmup 80 sec latency 333.163 ms 2024-09-19T06:19:36.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:36 smithi155 ceph-mon[35292]: pgmap v303: 129 pgs: 129 active+clean; 44 MiB data, 413 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.9 MiB/s wr, 80 op/s 2024-09-19T06:19:36.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:36 smithi053 ceph-mon[27968]: pgmap v303: 129 pgs: 129 active+clean; 44 MiB data, 413 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.9 MiB/s wr, 80 op/s 2024-09-19T06:19:37.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 64094 4.34 MB/sec warmup 84 sec latency 10.654 ms 2024-09-19T06:19:37.687 INFO:tasks.workunit.client.0.smithi053.stdout: 1 196736 13.31 MB/sec warmup 81 sec latency 182.752 ms 2024-09-19T06:19:38.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 64607 4.30 MB/sec warmup 85 sec latency 18.402 ms 2024-09-19T06:19:38.687 INFO:tasks.workunit.client.0.smithi053.stdout: 1 200127 13.38 MB/sec warmup 82 sec latency 41.920 ms 2024-09-19T06:19:38.729 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:38 smithi155 ceph-mon[35292]: pgmap v304: 129 pgs: 129 active+clean; 45 MiB data, 417 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.7 MiB/s wr, 95 op/s 2024-09-19T06:19:38.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:38 smithi053 ceph-mon[27968]: pgmap v304: 129 pgs: 129 active+clean; 45 MiB data, 417 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.7 MiB/s wr, 95 op/s 2024-09-19T06:19:39.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 65399 4.31 MB/sec warmup 86 sec latency 12.648 ms 2024-09-19T06:19:39.687 INFO:tasks.workunit.client.0.smithi053.stdout: 1 203072 13.39 MB/sec warmup 83 sec latency 74.979 ms 2024-09-19T06:19:40.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 66239 4.28 MB/sec warmup 87 sec latency 14.340 ms 2024-09-19T06:19:40.687 INFO:tasks.workunit.client.0.smithi053.stdout: 1 206535 13.46 MB/sec warmup 84 sec latency 74.860 ms 2024-09-19T06:19:40.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:40 smithi155 ceph-mon[35292]: pgmap v305: 129 pgs: 129 active+clean; 46 MiB data, 426 MiB used, 715 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.1 MiB/s wr, 100 op/s 2024-09-19T06:19:40.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:40 smithi053 ceph-mon[27968]: pgmap v305: 129 pgs: 129 active+clean; 46 MiB data, 426 MiB used, 715 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.1 MiB/s wr, 100 op/s 2024-09-19T06:19:41.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 67177 4.31 MB/sec warmup 88 sec latency 18.075 ms 2024-09-19T06:19:41.688 INFO:tasks.workunit.client.0.smithi053.stdout: 1 210023 13.52 MB/sec warmup 85 sec latency 27.172 ms 2024-09-19T06:19:42.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 67793 4.32 MB/sec warmup 89 sec latency 13.677 ms 2024-09-19T06:19:42.688 INFO:tasks.workunit.client.0.smithi053.stdout: 1 213556 13.59 MB/sec warmup 86 sec latency 30.605 ms 2024-09-19T06:19:42.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:42 smithi155 ceph-mon[35292]: pgmap v306: 129 pgs: 129 active+clean; 46 MiB data, 426 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 MiB/s wr, 73 op/s 2024-09-19T06:19:42.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:42 smithi053 ceph-mon[27968]: pgmap v306: 129 pgs: 129 active+clean; 46 MiB data, 426 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 MiB/s wr, 73 op/s 2024-09-19T06:19:43.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 68407 4.28 MB/sec warmup 90 sec latency 21.473 ms 2024-09-19T06:19:43.687 INFO:tasks.workunit.client.0.smithi053.stdout: 1 217104 13.65 MB/sec warmup 87 sec latency 49.231 ms 2024-09-19T06:19:44.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 69252 4.28 MB/sec warmup 91 sec latency 16.966 ms 2024-09-19T06:19:44.688 INFO:tasks.workunit.client.0.smithi053.stdout: 1 219748 13.66 MB/sec warmup 88 sec latency 39.892 ms 2024-09-19T06:19:44.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:44 smithi155 ceph-mon[35292]: pgmap v307: 129 pgs: 129 active+clean; 49 MiB data, 429 MiB used, 715 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.1 MiB/s wr, 102 op/s 2024-09-19T06:19:44.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:44 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:19:44.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:44 smithi053 ceph-mon[27968]: pgmap v307: 129 pgs: 129 active+clean; 49 MiB data, 429 MiB used, 715 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.1 MiB/s wr, 102 op/s 2024-09-19T06:19:44.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:44 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:19:45.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 70021 4.26 MB/sec warmup 92 sec latency 12.766 ms 2024-09-19T06:19:45.688 INFO:tasks.workunit.client.0.smithi053.stdout: 1 222962 13.70 MB/sec warmup 89 sec latency 26.425 ms 2024-09-19T06:19:46.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 70945 4.32 MB/sec warmup 93 sec latency 12.339 ms 2024-09-19T06:19:46.688 INFO:tasks.workunit.client.0.smithi053.stdout: 1 225992 13.71 MB/sec warmup 90 sec latency 33.573 ms 2024-09-19T06:19:46.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:46 smithi155 ceph-mon[35292]: pgmap v308: 129 pgs: 129 active+clean; 47 MiB data, 457 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.9 MiB/s wr, 86 op/s 2024-09-19T06:19:46.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:46 smithi053 ceph-mon[27968]: pgmap v308: 129 pgs: 129 active+clean; 47 MiB data, 457 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.9 MiB/s wr, 86 op/s 2024-09-19T06:19:47.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 71545 4.29 MB/sec warmup 94 sec latency 11.175 ms 2024-09-19T06:19:47.688 INFO:tasks.workunit.client.0.smithi053.stdout: 1 228948 13.74 MB/sec warmup 91 sec latency 57.798 ms 2024-09-19T06:19:48.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 72351 4.30 MB/sec warmup 95 sec latency 15.983 ms 2024-09-19T06:19:48.688 INFO:tasks.workunit.client.0.smithi053.stdout: 1 231531 13.75 MB/sec warmup 92 sec latency 86.117 ms 2024-09-19T06:19:48.729 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:48 smithi155 ceph-mon[35292]: pgmap v309: 129 pgs: 129 active+clean; 48 MiB data, 458 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.8 MiB/s wr, 89 op/s 2024-09-19T06:19:48.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:48 smithi053 ceph-mon[27968]: pgmap v309: 129 pgs: 129 active+clean; 48 MiB data, 458 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.8 MiB/s wr, 89 op/s 2024-09-19T06:19:49.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 73182 4.26 MB/sec warmup 96 sec latency 12.398 ms 2024-09-19T06:19:49.688 INFO:tasks.workunit.client.0.smithi053.stdout: 1 234957 13.80 MB/sec warmup 93 sec latency 50.023 ms 2024-09-19T06:19:50.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 74225 4.30 MB/sec warmup 97 sec latency 12.993 ms 2024-09-19T06:19:50.688 INFO:tasks.workunit.client.0.smithi053.stdout: 1 238436 13.86 MB/sec warmup 94 sec latency 73.840 ms 2024-09-19T06:19:50.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:50 smithi155 ceph-mon[35292]: pgmap v310: 129 pgs: 129 active+clean; 50 MiB data, 466 MiB used, 715 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.1 MiB/s wr, 97 op/s 2024-09-19T06:19:50.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:50 smithi053 ceph-mon[27968]: pgmap v310: 129 pgs: 129 active+clean; 50 MiB data, 466 MiB used, 715 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.1 MiB/s wr, 97 op/s 2024-09-19T06:19:51.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 74805 4.31 MB/sec warmup 98 sec latency 11.811 ms 2024-09-19T06:19:51.688 INFO:tasks.workunit.client.0.smithi053.stdout: 1 241927 13.91 MB/sec warmup 95 sec latency 33.367 ms 2024-09-19T06:19:52.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 75372 4.27 MB/sec warmup 99 sec latency 15.659 ms 2024-09-19T06:19:52.688 INFO:tasks.workunit.client.0.smithi053.stdout: 1 245465 13.97 MB/sec warmup 96 sec latency 30.619 ms 2024-09-19T06:19:52.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:52 smithi155 ceph-mon[35292]: pgmap v311: 129 pgs: 129 active+clean; 50 MiB data, 465 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 86 op/s 2024-09-19T06:19:52.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:52 smithi053 ceph-mon[27968]: pgmap v311: 129 pgs: 129 active+clean; 50 MiB data, 465 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 86 op/s 2024-09-19T06:19:53.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 76190 4.28 MB/sec warmup 100 sec latency 12.881 ms 2024-09-19T06:19:53.688 INFO:tasks.workunit.client.0.smithi053.stdout: 1 248101 13.97 MB/sec warmup 97 sec latency 41.294 ms 2024-09-19T06:19:54.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 76998 4.26 MB/sec warmup 101 sec latency 19.651 ms 2024-09-19T06:19:54.688 INFO:tasks.workunit.client.0.smithi053.stdout: 1 251248 14.00 MB/sec warmup 98 sec latency 49.864 ms 2024-09-19T06:19:54.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:54 smithi155 ceph-mon[35292]: pgmap v312: 129 pgs: 129 active+clean; 50 MiB data, 483 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 1.9 MiB/s wr, 96 op/s 2024-09-19T06:19:54.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:54 smithi053 ceph-mon[27968]: pgmap v312: 129 pgs: 129 active+clean; 50 MiB data, 483 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 1.9 MiB/s wr, 96 op/s 2024-09-19T06:19:55.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 77868 4.28 MB/sec warmup 102 sec latency 12.852 ms 2024-09-19T06:19:55.688 INFO:tasks.workunit.client.0.smithi053.stdout: 1 254226 14.00 MB/sec warmup 99 sec latency 41.408 ms 2024-09-19T06:19:56.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 78535 4.29 MB/sec warmup 103 sec latency 12.975 ms 2024-09-19T06:19:56.688 INFO:tasks.workunit.client.0.smithi053.stdout: 1 257061 14.03 MB/sec warmup 100 sec latency 33.211 ms 2024-09-19T06:19:56.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:56 smithi155 ceph-mon[35292]: pgmap v313: 129 pgs: 129 active+clean; 51 MiB data, 475 MiB used, 715 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.9 MiB/s wr, 94 op/s 2024-09-19T06:19:56.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:56 smithi053 ceph-mon[27968]: pgmap v313: 129 pgs: 129 active+clean; 51 MiB data, 475 MiB used, 715 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.9 MiB/s wr, 94 op/s 2024-09-19T06:19:57.618 INFO:tasks.workunit.client.1.smithi155.stdout: 1 79262 4.27 MB/sec warmup 104 sec latency 12.652 ms 2024-09-19T06:19:57.688 INFO:tasks.workunit.client.0.smithi053.stdout: 1 259856 14.04 MB/sec warmup 101 sec latency 56.108 ms 2024-09-19T06:19:58.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 80091 4.26 MB/sec warmup 105 sec latency 13.161 ms 2024-09-19T06:19:58.688 INFO:tasks.workunit.client.0.smithi053.stdout: 1 263313 14.08 MB/sec warmup 102 sec latency 116.779 ms 2024-09-19T06:19:58.730 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:58 smithi155 ceph-mon[35292]: pgmap v314: 129 pgs: 129 active+clean; 51 MiB data, 479 MiB used, 715 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.8 MiB/s wr, 91 op/s 2024-09-19T06:19:58.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:58 smithi053 ceph-mon[27968]: pgmap v314: 129 pgs: 129 active+clean; 51 MiB data, 479 MiB used, 715 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.8 MiB/s wr, 91 op/s 2024-09-19T06:19:59.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 80986 4.28 MB/sec warmup 106 sec latency 12.887 ms 2024-09-19T06:19:59.688 INFO:tasks.workunit.client.0.smithi053.stdout: 1 266798 14.13 MB/sec warmup 103 sec latency 46.219 ms 2024-09-19T06:19:59.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:19:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:19:59.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:19:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:20:00.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 81769 4.29 MB/sec warmup 107 sec latency 13.515 ms 2024-09-19T06:20:00.688 INFO:tasks.workunit.client.0.smithi053.stdout: 1 269362 14.13 MB/sec warmup 104 sec latency 125.272 ms 2024-09-19T06:20:00.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:00 smithi155 ceph-mon[35292]: pgmap v315: 129 pgs: 129 active+clean; 52 MiB data, 492 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 2.1 MiB/s wr, 108 op/s 2024-09-19T06:20:00.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:00 smithi155 ceph-mon[35292]: overall HEALTH_OK 2024-09-19T06:20:00.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:00 smithi053 ceph-mon[27968]: pgmap v315: 129 pgs: 129 active+clean; 52 MiB data, 492 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 2.1 MiB/s wr, 108 op/s 2024-09-19T06:20:00.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:00 smithi053 ceph-mon[27968]: overall HEALTH_OK 2024-09-19T06:20:01.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 82317 4.27 MB/sec warmup 108 sec latency 19.746 ms 2024-09-19T06:20:01.688 INFO:tasks.workunit.client.0.smithi053.stdout: 1 272522 14.16 MB/sec warmup 105 sec latency 50.108 ms 2024-09-19T06:20:01.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:01 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:20:01.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:01 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:20:02.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 83131 4.27 MB/sec warmup 109 sec latency 15.427 ms 2024-09-19T06:20:02.688 INFO:tasks.workunit.client.0.smithi053.stdout: 1 274075 14.10 MB/sec warmup 106 sec latency 208.177 ms 2024-09-19T06:20:02.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:02 smithi155 ceph-mon[35292]: pgmap v316: 129 pgs: 129 active+clean; 52 MiB data, 492 MiB used, 715 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.6 MiB/s wr, 79 op/s 2024-09-19T06:20:02.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:02 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:20:02.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:02 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:20:02.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:02 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:20:02.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:02 smithi053 ceph-mon[27968]: pgmap v316: 129 pgs: 129 active+clean; 52 MiB data, 492 MiB used, 715 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.6 MiB/s wr, 79 op/s 2024-09-19T06:20:02.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:02 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:20:02.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:02 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:20:02.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:02 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:20:03.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 83999 4.25 MB/sec warmup 110 sec latency 18.757 ms 2024-09-19T06:20:03.688 INFO:tasks.workunit.client.0.smithi053.stdout: 1 277577 14.14 MB/sec warmup 107 sec latency 27.002 ms 2024-09-19T06:20:04.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 84938 4.28 MB/sec warmup 111 sec latency 15.874 ms 2024-09-19T06:20:04.688 INFO:tasks.workunit.client.0.smithi053.stdout: 1 281050 14.19 MB/sec warmup 108 sec latency 50.125 ms 2024-09-19T06:20:04.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:04 smithi155 ceph-mon[35292]: pgmap v317: 129 pgs: 129 active+clean; 52 MiB data, 483 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 1.8 MiB/s wr, 90 op/s 2024-09-19T06:20:04.776 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:04 smithi053 ceph-mon[27968]: pgmap v317: 129 pgs: 129 active+clean; 52 MiB data, 483 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 1.8 MiB/s wr, 90 op/s 2024-09-19T06:20:05.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 85553 4.29 MB/sec warmup 112 sec latency 12.064 ms 2024-09-19T06:20:05.688 INFO:tasks.workunit.client.0.smithi053.stdout: 1 284470 14.23 MB/sec warmup 109 sec latency 141.503 ms 2024-09-19T06:20:06.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 86155 4.26 MB/sec warmup 113 sec latency 17.072 ms 2024-09-19T06:20:06.688 INFO:tasks.workunit.client.0.smithi053.stdout: 1 288011 14.27 MB/sec warmup 110 sec latency 52.203 ms 2024-09-19T06:20:06.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:06 smithi155 ceph-mon[35292]: pgmap v318: 129 pgs: 129 active+clean; 54 MiB data, 488 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 2.0 MiB/s wr, 106 op/s 2024-09-19T06:20:06.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:06 smithi053 ceph-mon[27968]: pgmap v318: 129 pgs: 129 active+clean; 54 MiB data, 488 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 2.0 MiB/s wr, 106 op/s 2024-09-19T06:20:07.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 86988 4.26 MB/sec warmup 114 sec latency 14.940 ms 2024-09-19T06:20:07.688 INFO:tasks.workunit.client.0.smithi053.stdout: 1 290669 14.27 MB/sec warmup 111 sec latency 33.263 ms 2024-09-19T06:20:08.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 87756 4.24 MB/sec warmup 115 sec latency 16.516 ms 2024-09-19T06:20:08.688 INFO:tasks.workunit.client.0.smithi053.stdout: 1 291617 14.19 MB/sec warmup 112 sec latency 275.116 ms 2024-09-19T06:20:08.814 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:08 smithi155 ceph-mon[35292]: pgmap v319: 129 pgs: 129 active+clean; 54 MiB data, 489 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.7 MiB/s wr, 83 op/s 2024-09-19T06:20:08.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:08 smithi053 ceph-mon[27968]: pgmap v319: 129 pgs: 129 active+clean; 54 MiB data, 489 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.7 MiB/s wr, 83 op/s 2024-09-19T06:20:09.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 88507 4.26 MB/sec warmup 116 sec latency 67.244 ms 2024-09-19T06:20:09.688 INFO:tasks.workunit.client.0.smithi053.stdout: 1 291678 14.07 MB/sec warmup 113 sec latency 308.543 ms 2024-09-19T06:20:10.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 89156 4.27 MB/sec warmup 117 sec latency 45.476 ms 2024-09-19T06:20:10.688 INFO:tasks.workunit.client.0.smithi053.stdout: 1 291688 13.94 MB/sec warmup 114 sec latency 308.493 ms 2024-09-19T06:20:10.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:10 smithi053 ceph-mon[27968]: pgmap v320: 129 pgs: 129 active+clean; 54 MiB data, 492 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.1 MiB/s wr, 107 op/s 2024-09-19T06:20:10.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:10 smithi155 ceph-mon[35292]: pgmap v320: 129 pgs: 129 active+clean; 54 MiB data, 492 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.1 MiB/s wr, 107 op/s 2024-09-19T06:20:11.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 89835 4.25 MB/sec warmup 118 sec latency 26.276 ms 2024-09-19T06:20:11.689 INFO:tasks.workunit.client.0.smithi053.stdout: 1 291799 13.82 MB/sec warmup 115 sec latency 300.347 ms 2024-09-19T06:20:12.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 90573 4.24 MB/sec warmup 119 sec latency 89.321 ms 2024-09-19T06:20:12.689 INFO:tasks.workunit.client.0.smithi053.stdout: 1 292363 13.74 MB/sec warmup 116 sec latency 341.626 ms 2024-09-19T06:20:12.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:12 smithi053 ceph-mon[27968]: pgmap v321: 129 pgs: 129 active+clean; 54 MiB data, 492 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.6 MiB/s wr, 81 op/s 2024-09-19T06:20:12.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:12 smithi155 ceph-mon[35292]: pgmap v321: 129 pgs: 129 active+clean; 54 MiB data, 492 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.6 MiB/s wr, 81 op/s 2024-09-19T06:20:13.689 INFO:tasks.workunit.client.0.smithi053.stdout: 1 292765 13.63 MB/sec warmup 117 sec latency 325.192 ms 2024-09-19T06:20:14.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 92054 6.37 MB/sec execute 1 sec latency 89.098 ms 2024-09-19T06:20:14.689 INFO:tasks.workunit.client.0.smithi053.stdout: 1 293252 13.53 MB/sec warmup 118 sec latency 291.898 ms 2024-09-19T06:20:14.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:14 smithi053 ceph-mon[27968]: pgmap v322: 129 pgs: 129 active+clean; 55 MiB data, 497 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 1.9 MiB/s wr, 90 op/s 2024-09-19T06:20:14.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:14 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:20:14.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:14 smithi155 ceph-mon[35292]: pgmap v322: 129 pgs: 129 active+clean; 55 MiB data, 497 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 1.9 MiB/s wr, 90 op/s 2024-09-19T06:20:14.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:14 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:20:15.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 92717 5.53 MB/sec execute 2 sec latency 29.322 ms 2024-09-19T06:20:15.689 INFO:tasks.workunit.client.0.smithi053.stdout: 1 293641 13.45 MB/sec warmup 119 sec latency 313.287 ms 2024-09-19T06:20:16.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 93307 4.33 MB/sec execute 3 sec latency 111.082 ms 2024-09-19T06:20:16.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:16 smithi053 ceph-mon[27968]: pgmap v323: 129 pgs: 129 active+clean; 56 MiB data, 500 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 2.0 MiB/s wr, 102 op/s 2024-09-19T06:20:16.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:16 smithi155 ceph-mon[35292]: pgmap v323: 129 pgs: 129 active+clean; 56 MiB data, 500 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 2.0 MiB/s wr, 102 op/s 2024-09-19T06:20:17.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 94146 4.12 MB/sec execute 4 sec latency 14.784 ms 2024-09-19T06:20:17.689 INFO:tasks.workunit.client.0.smithi053.stdout: 1 297757 19.14 MB/sec execute 1 sec latency 301.836 ms 2024-09-19T06:20:18.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 95020 4.41 MB/sec execute 5 sec latency 13.841 ms 2024-09-19T06:20:18.689 INFO:tasks.workunit.client.0.smithi053.stdout: 1 300972 18.21 MB/sec execute 2 sec latency 93.490 ms 2024-09-19T06:20:18.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:18 smithi053 ceph-mon[27968]: pgmap v324: 129 pgs: 129 active+clean; 56 MiB data, 518 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.6 MiB/s wr, 81 op/s 2024-09-19T06:20:18.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:18 smithi155 ceph-mon[35292]: pgmap v324: 129 pgs: 129 active+clean; 56 MiB data, 518 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.6 MiB/s wr, 81 op/s 2024-09-19T06:20:19.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 95886 4.79 MB/sec execute 6 sec latency 12.373 ms 2024-09-19T06:20:19.689 INFO:tasks.workunit.client.0.smithi053.stdout: 1 303414 16.13 MB/sec execute 3 sec latency 125.149 ms 2024-09-19T06:20:20.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 96433 4.32 MB/sec execute 7 sec latency 16.195 ms 2024-09-19T06:20:20.689 INFO:tasks.workunit.client.0.smithi053.stdout: 1 306703 16.82 MB/sec execute 4 sec latency 33.698 ms 2024-09-19T06:20:20.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:20 smithi053 ceph-mon[27968]: pgmap v325: 129 pgs: 129 active+clean; 58 MiB data, 548 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.0 MiB/s wr, 103 op/s 2024-09-19T06:20:20.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:20 smithi155 ceph-mon[35292]: pgmap v325: 129 pgs: 129 active+clean; 58 MiB data, 548 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.0 MiB/s wr, 103 op/s 2024-09-19T06:20:21.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 97226 4.41 MB/sec execute 8 sec latency 15.586 ms 2024-09-19T06:20:21.689 INFO:tasks.workunit.client.0.smithi053.stdout: 1 309418 16.34 MB/sec execute 5 sec latency 115.059 ms 2024-09-19T06:20:22.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 98039 4.00 MB/sec execute 9 sec latency 13.676 ms 2024-09-19T06:20:22.689 INFO:tasks.workunit.client.0.smithi053.stdout: 1 312893 16.78 MB/sec execute 6 sec latency 41.727 ms 2024-09-19T06:20:22.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:22 smithi053 ceph-mon[27968]: pgmap v326: 129 pgs: 129 active+clean; 57 MiB data, 547 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.5 MiB/s wr, 75 op/s 2024-09-19T06:20:22.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:22 smithi155 ceph-mon[35292]: pgmap v326: 129 pgs: 129 active+clean; 57 MiB data, 547 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.5 MiB/s wr, 75 op/s 2024-09-19T06:20:23.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 99094 4.46 MB/sec execute 10 sec latency 14.585 ms 2024-09-19T06:20:23.689 INFO:tasks.workunit.client.0.smithi053.stdout: 1 316437 17.11 MB/sec execute 7 sec latency 33.365 ms 2024-09-19T06:20:24.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 99659 4.46 MB/sec execute 11 sec latency 12.220 ms 2024-09-19T06:20:24.689 INFO:tasks.workunit.client.0.smithi053.stdout: 1 319930 17.35 MB/sec execute 8 sec latency 50.169 ms 2024-09-19T06:20:24.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:24 smithi053 ceph-mon[27968]: pgmap v327: 129 pgs: 129 active+clean; 58 MiB data, 554 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 1.8 MiB/s wr, 85 op/s 2024-09-19T06:20:24.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:24 smithi155 ceph-mon[35292]: pgmap v327: 129 pgs: 129 active+clean; 58 MiB data, 554 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 1.8 MiB/s wr, 85 op/s 2024-09-19T06:20:25.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 100282 4.18 MB/sec execute 12 sec latency 15.253 ms 2024-09-19T06:20:25.689 INFO:tasks.workunit.client.0.smithi053.stdout: 1 322583 17.00 MB/sec execute 9 sec latency 34.698 ms 2024-09-19T06:20:26.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 101086 4.20 MB/sec execute 13 sec latency 12.822 ms 2024-09-19T06:20:26.689 INFO:tasks.workunit.client.0.smithi053.stdout: 1 325561 16.87 MB/sec execute 10 sec latency 100.317 ms 2024-09-19T06:20:26.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:26 smithi053 ceph-mon[27968]: pgmap v328: 129 pgs: 129 active+clean; 60 MiB data, 560 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.9 MiB/s wr, 100 op/s 2024-09-19T06:20:26.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:26 smithi155 ceph-mon[35292]: pgmap v328: 129 pgs: 129 active+clean; 60 MiB data, 560 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.9 MiB/s wr, 100 op/s 2024-09-19T06:20:27.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 101909 4.09 MB/sec execute 14 sec latency 11.741 ms 2024-09-19T06:20:27.689 INFO:tasks.workunit.client.0.smithi053.stdout: 1 328232 16.57 MB/sec execute 11 sec latency 34.809 ms 2024-09-19T06:20:28.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 102694 4.25 MB/sec execute 15 sec latency 13.259 ms 2024-09-19T06:20:28.689 INFO:tasks.workunit.client.0.smithi053.stdout: 1 331523 16.76 MB/sec execute 12 sec latency 33.654 ms 2024-09-19T06:20:28.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:28 smithi053 ceph-mon[27968]: pgmap v329: 129 pgs: 129 active+clean; 60 MiB data, 566 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.7 MiB/s wr, 81 op/s 2024-09-19T06:20:28.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:28 smithi155 ceph-mon[35292]: pgmap v329: 129 pgs: 129 active+clean; 60 MiB data, 566 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.7 MiB/s wr, 81 op/s 2024-09-19T06:20:29.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 103375 4.28 MB/sec execute 16 sec latency 11.769 ms 2024-09-19T06:20:29.689 INFO:tasks.workunit.client.0.smithi053.stdout: 1 334315 16.60 MB/sec execute 13 sec latency 53.217 ms 2024-09-19T06:20:29.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:20:29.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:29 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:20:30.619 INFO:tasks.workunit.client.1.smithi155.stdout: 1 104086 4.15 MB/sec execute 17 sec latency 16.609 ms 2024-09-19T06:20:30.689 INFO:tasks.workunit.client.0.smithi053.stdout: 1 337784 16.76 MB/sec execute 14 sec latency 32.699 ms 2024-09-19T06:20:30.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:30 smithi053 ceph-mon[27968]: pgmap v330: 129 pgs: 129 active+clean; 60 MiB data, 566 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 2.1 MiB/s wr, 105 op/s 2024-09-19T06:20:30.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:30 smithi155 ceph-mon[35292]: pgmap v330: 129 pgs: 129 active+clean; 60 MiB data, 566 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 2.1 MiB/s wr, 105 op/s 2024-09-19T06:20:31.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 104947 4.12 MB/sec execute 18 sec latency 15.056 ms 2024-09-19T06:20:31.689 INFO:tasks.workunit.client.0.smithi053.stdout: 1 341260 16.91 MB/sec execute 15 sec latency 49.614 ms 2024-09-19T06:20:32.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 105829 4.26 MB/sec execute 19 sec latency 17.758 ms 2024-09-19T06:20:32.689 INFO:tasks.workunit.client.0.smithi053.stdout: 1 344805 17.05 MB/sec execute 16 sec latency 33.733 ms 2024-09-19T06:20:32.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:32 smithi053 ceph-mon[27968]: pgmap v331: 129 pgs: 129 active+clean; 61 MiB data, 566 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.7 MiB/s wr, 80 op/s 2024-09-19T06:20:32.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:32 smithi155 ceph-mon[35292]: pgmap v331: 129 pgs: 129 active+clean; 61 MiB data, 566 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.7 MiB/s wr, 80 op/s 2024-09-19T06:20:33.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 106582 4.31 MB/sec execute 20 sec latency 12.231 ms 2024-09-19T06:20:33.689 INFO:tasks.workunit.client.0.smithi053.stdout: 1 348292 17.16 MB/sec execute 17 sec latency 58.458 ms 2024-09-19T06:20:34.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:34 smithi053 ceph-mon[27968]: pgmap v332: 129 pgs: 129 active+clean; 61 MiB data, 573 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 2.1 MiB/s wr, 106 op/s 2024-09-19T06:20:34.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 107115 4.18 MB/sec execute 21 sec latency 15.047 ms 2024-09-19T06:20:34.690 INFO:tasks.workunit.client.0.smithi053.stdout: 1 350936 17.00 MB/sec execute 18 sec latency 40.614 ms 2024-09-19T06:20:34.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:34 smithi155 ceph-mon[35292]: pgmap v332: 129 pgs: 129 active+clean; 61 MiB data, 573 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 2.1 MiB/s wr, 106 op/s 2024-09-19T06:20:35.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 107945 4.21 MB/sec execute 22 sec latency 12.259 ms 2024-09-19T06:20:35.690 INFO:tasks.workunit.client.0.smithi053.stdout: 1 353925 16.93 MB/sec execute 19 sec latency 133.616 ms 2024-09-19T06:20:36.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 108813 4.11 MB/sec execute 23 sec latency 15.349 ms 2024-09-19T06:20:36.690 INFO:tasks.workunit.client.0.smithi053.stdout: 1 357046 16.83 MB/sec execute 20 sec latency 39.912 ms 2024-09-19T06:20:36.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:36 smithi053 ceph-mon[27968]: pgmap v333: 129 pgs: 129 active+clean; 62 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 3.6 KiB/s rd, 2.1 MiB/s wr, 104 op/s 2024-09-19T06:20:36.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:36 smithi155 ceph-mon[35292]: pgmap v333: 129 pgs: 129 active+clean; 62 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 3.6 KiB/s rd, 2.1 MiB/s wr, 104 op/s 2024-09-19T06:20:37.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 109759 4.25 MB/sec execute 24 sec latency 17.391 ms 2024-09-19T06:20:37.690 INFO:tasks.workunit.client.0.smithi053.stdout: 1 360142 16.87 MB/sec execute 21 sec latency 32.702 ms 2024-09-19T06:20:38.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 110399 4.27 MB/sec execute 25 sec latency 10.847 ms 2024-09-19T06:20:38.690 INFO:tasks.workunit.client.0.smithi053.stdout: 1 362784 16.77 MB/sec execute 22 sec latency 90.863 ms 2024-09-19T06:20:38.783 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:38 smithi155 ceph-mon[35292]: pgmap v334: 129 pgs: 129 active+clean; 62 MiB data, 574 MiB used, 715 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.7 MiB/s wr, 82 op/s 2024-09-19T06:20:38.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:38 smithi053 ceph-mon[27968]: pgmap v334: 129 pgs: 129 active+clean; 62 MiB data, 574 MiB used, 715 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.7 MiB/s wr, 82 op/s 2024-09-19T06:20:39.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 111064 4.18 MB/sec execute 26 sec latency 14.430 ms 2024-09-19T06:20:39.690 INFO:tasks.workunit.client.0.smithi053.stdout: 1 366226 16.86 MB/sec execute 23 sec latency 116.805 ms 2024-09-19T06:20:40.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 111916 4.16 MB/sec execute 27 sec latency 14.256 ms 2024-09-19T06:20:40.690 INFO:tasks.workunit.client.0.smithi053.stdout: 1 369691 16.95 MB/sec execute 24 sec latency 41.589 ms 2024-09-19T06:20:40.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:40 smithi053 ceph-mon[27968]: pgmap v335: 129 pgs: 129 active+clean; 62 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.1 MiB/s wr, 108 op/s 2024-09-19T06:20:40.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:40 smithi155 ceph-mon[35292]: pgmap v335: 129 pgs: 129 active+clean; 62 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.1 MiB/s wr, 108 op/s 2024-09-19T06:20:41.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 112868 4.21 MB/sec execute 28 sec latency 13.039 ms 2024-09-19T06:20:41.690 INFO:tasks.workunit.client.0.smithi053.stdout: 1 373169 17.03 MB/sec execute 25 sec latency 74.307 ms 2024-09-19T06:20:42.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 113563 4.28 MB/sec execute 29 sec latency 32.523 ms 2024-09-19T06:20:42.690 INFO:tasks.workunit.client.0.smithi053.stdout: 1 375411 16.84 MB/sec execute 26 sec latency 198.738 ms 2024-09-19T06:20:42.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:42 smithi053 ceph-mon[27968]: pgmap v336: 129 pgs: 129 active+clean; 63 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 MiB/s wr, 80 op/s 2024-09-19T06:20:42.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:42 smithi155 ceph-mon[35292]: pgmap v336: 129 pgs: 129 active+clean; 63 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 MiB/s wr, 80 op/s 2024-09-19T06:20:43.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 114154 4.19 MB/sec execute 30 sec latency 12.013 ms 2024-09-19T06:20:43.690 INFO:tasks.workunit.client.0.smithi053.stdout: 1 378391 16.72 MB/sec execute 27 sec latency 50.001 ms 2024-09-19T06:20:44.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 114967 4.22 MB/sec execute 31 sec latency 16.579 ms 2024-09-19T06:20:44.690 INFO:tasks.workunit.client.0.smithi053.stdout: 1 381454 16.75 MB/sec execute 28 sec latency 33.576 ms 2024-09-19T06:20:44.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:44 smithi053 ceph-mon[27968]: pgmap v337: 129 pgs: 129 active+clean; 64 MiB data, 587 MiB used, 715 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.9 MiB/s wr, 103 op/s 2024-09-19T06:20:44.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:44 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:20:44.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:44 smithi155 ceph-mon[35292]: pgmap v337: 129 pgs: 129 active+clean; 64 MiB data, 587 MiB used, 715 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.9 MiB/s wr, 103 op/s 2024-09-19T06:20:44.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:44 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:20:45.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 115852 4.14 MB/sec execute 32 sec latency 15.107 ms 2024-09-19T06:20:45.690 INFO:tasks.workunit.client.0.smithi053.stdout: 1 384097 16.68 MB/sec execute 29 sec latency 32.114 ms 2024-09-19T06:20:46.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 116825 4.25 MB/sec execute 33 sec latency 18.795 ms 2024-09-19T06:20:46.690 INFO:tasks.workunit.client.0.smithi053.stdout: 1 387560 16.75 MB/sec execute 30 sec latency 66.767 ms 2024-09-19T06:20:46.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:46 smithi053 ceph-mon[27968]: pgmap v338: 129 pgs: 129 active+clean; 65 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.0 MiB/s wr, 96 op/s 2024-09-19T06:20:46.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:46 smithi155 ceph-mon[35292]: pgmap v338: 129 pgs: 129 active+clean; 65 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.0 MiB/s wr, 96 op/s 2024-09-19T06:20:47.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 117386 4.25 MB/sec execute 34 sec latency 12.865 ms 2024-09-19T06:20:47.690 INFO:tasks.workunit.client.0.smithi053.stdout: 1 391022 16.82 MB/sec execute 31 sec latency 109.038 ms 2024-09-19T06:20:48.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 117973 4.16 MB/sec execute 35 sec latency 18.785 ms 2024-09-19T06:20:48.690 INFO:tasks.workunit.client.0.smithi053.stdout: 1 391146 16.31 MB/sec execute 32 sec latency 275.245 ms 2024-09-19T06:20:48.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:48 smithi053 ceph-mon[27968]: pgmap v339: 129 pgs: 129 active+clean; 65 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.8 MiB/s wr, 92 op/s 2024-09-19T06:20:48.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:48 smithi155 ceph-mon[35292]: pgmap v339: 129 pgs: 129 active+clean; 65 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.8 MiB/s wr, 92 op/s 2024-09-19T06:20:49.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 118731 4.17 MB/sec execute 36 sec latency 40.632 ms 2024-09-19T06:20:49.690 INFO:tasks.workunit.client.0.smithi053.stdout: 1 391857 15.95 MB/sec execute 33 sec latency 308.078 ms 2024-09-19T06:20:50.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 119480 4.11 MB/sec execute 37 sec latency 74.965 ms 2024-09-19T06:20:50.690 INFO:tasks.workunit.client.0.smithi053.stdout: 1 392112 15.48 MB/sec execute 34 sec latency 332.239 ms 2024-09-19T06:20:50.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:50 smithi053 ceph-mon[27968]: pgmap v340: 129 pgs: 129 active+clean; 65 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.1 MiB/s wr, 100 op/s 2024-09-19T06:20:50.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:50 smithi155 ceph-mon[35292]: pgmap v340: 129 pgs: 129 active+clean; 65 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.1 MiB/s wr, 100 op/s 2024-09-19T06:20:51.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 120396 4.19 MB/sec execute 38 sec latency 30.587 ms 2024-09-19T06:20:51.690 INFO:tasks.workunit.client.0.smithi053.stdout: 1 392597 15.09 MB/sec execute 35 sec latency 350.275 ms 2024-09-19T06:20:52.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 120924 4.20 MB/sec execute 39 sec latency 85.861 ms 2024-09-19T06:20:52.690 INFO:tasks.workunit.client.0.smithi053.stdout: 1 393138 14.82 MB/sec execute 36 sec latency 383.355 ms 2024-09-19T06:20:52.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:52 smithi053 ceph-mon[27968]: pgmap v341: 129 pgs: 129 active+clean; 66 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.7 MiB/s wr, 86 op/s 2024-09-19T06:20:52.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:52 smithi155 ceph-mon[35292]: pgmap v341: 129 pgs: 129 active+clean; 66 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.7 MiB/s wr, 86 op/s 2024-09-19T06:20:53.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 121424 4.11 MB/sec execute 40 sec latency 51.238 ms 2024-09-19T06:20:53.690 INFO:tasks.workunit.client.0.smithi053.stdout: 1 393413 14.44 MB/sec execute 37 sec latency 307.320 ms 2024-09-19T06:20:54.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 122169 4.13 MB/sec execute 41 sec latency 43.150 ms 2024-09-19T06:20:54.690 INFO:tasks.workunit.client.0.smithi053.stdout: 1 393545 14.09 MB/sec execute 38 sec latency 301.920 ms 2024-09-19T06:20:54.848 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:54 smithi053 ceph-mon[27968]: pgmap v342: 129 pgs: 129 active+clean; 67 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.9 MiB/s wr, 95 op/s 2024-09-19T06:20:54.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:54 smithi155 ceph-mon[35292]: pgmap v342: 129 pgs: 129 active+clean; 67 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.9 MiB/s wr, 95 op/s 2024-09-19T06:20:55.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 122941 4.07 MB/sec execute 42 sec latency 76.197 ms 2024-09-19T06:20:55.690 INFO:tasks.workunit.client.0.smithi053.stdout: 1 394448 13.86 MB/sec execute 39 sec latency 299.841 ms 2024-09-19T06:20:56.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 123910 4.15 MB/sec execute 43 sec latency 40.554 ms 2024-09-19T06:20:56.690 INFO:tasks.workunit.client.0.smithi053.stdout: 1 394501 13.51 MB/sec execute 40 sec latency 316.787 ms 2024-09-19T06:20:56.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:56 smithi053 ceph-mon[27968]: pgmap v343: 129 pgs: 129 active+clean; 67 MiB data, 605 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.0 MiB/s wr, 94 op/s 2024-09-19T06:20:56.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:56 smithi155 ceph-mon[35292]: pgmap v343: 129 pgs: 129 active+clean; 67 MiB data, 605 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.0 MiB/s wr, 94 op/s 2024-09-19T06:20:57.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 124477 4.16 MB/sec execute 44 sec latency 11.897 ms 2024-09-19T06:20:57.690 INFO:tasks.workunit.client.0.smithi053.stdout: 1 398042 13.65 MB/sec execute 41 sec latency 27.316 ms 2024-09-19T06:20:58.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 125073 4.09 MB/sec execute 45 sec latency 19.191 ms 2024-09-19T06:20:58.690 INFO:tasks.workunit.client.0.smithi053.stdout: 1 401540 13.78 MB/sec execute 42 sec latency 41.786 ms 2024-09-19T06:20:58.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:58 smithi053 ceph-mon[27968]: pgmap v344: 129 pgs: 129 active+clean; 67 MiB data, 605 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 81 op/s 2024-09-19T06:20:58.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:58 smithi155 ceph-mon[35292]: pgmap v344: 129 pgs: 129 active+clean; 67 MiB data, 605 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 81 op/s 2024-09-19T06:20:59.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 125899 4.10 MB/sec execute 46 sec latency 13.380 ms 2024-09-19T06:20:59.690 INFO:tasks.workunit.client.0.smithi053.stdout: 1 404184 13.79 MB/sec execute 43 sec latency 31.025 ms 2024-09-19T06:20:59.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:20:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:20:59.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:20:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:21:00.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 126682 4.06 MB/sec execute 47 sec latency 16.305 ms 2024-09-19T06:21:00.690 INFO:tasks.workunit.client.0.smithi053.stdout: 1 407380 13.87 MB/sec execute 44 sec latency 40.127 ms 2024-09-19T06:21:00.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:00 smithi053 ceph-mon[27968]: pgmap v345: 129 pgs: 129 active+clean; 67 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 4.8 KiB/s rd, 1.9 MiB/s wr, 90 op/s 2024-09-19T06:21:00.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:00 smithi155 ceph-mon[35292]: pgmap v345: 129 pgs: 129 active+clean; 67 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 4.8 KiB/s rd, 1.9 MiB/s wr, 90 op/s 2024-09-19T06:21:01.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 127519 4.12 MB/sec execute 48 sec latency 11.945 ms 2024-09-19T06:21:01.690 INFO:tasks.workunit.client.0.smithi053.stdout: 1 409841 13.82 MB/sec execute 45 sec latency 67.244 ms 2024-09-19T06:21:02.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 128134 4.13 MB/sec execute 49 sec latency 90.398 ms 2024-09-19T06:21:02.691 INFO:tasks.workunit.client.0.smithi053.stdout: 1 410366 13.56 MB/sec execute 46 sec latency 408.988 ms 2024-09-19T06:21:02.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:02 smithi053 ceph-mon[27968]: pgmap v346: 129 pgs: 129 active+clean; 68 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 3.1 KiB/s rd, 1.6 MiB/s wr, 85 op/s 2024-09-19T06:21:02.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:02 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:21:02.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:02 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:21:02.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:02 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:21:02.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:02 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:21:02.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:02 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:21:02.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:02 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:21:02.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:02 smithi155 ceph-mon[35292]: pgmap v346: 129 pgs: 129 active+clean; 68 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 3.1 KiB/s rd, 1.6 MiB/s wr, 85 op/s 2024-09-19T06:21:02.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:02 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:21:02.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:02 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:21:02.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:02 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:21:02.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:02 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:21:02.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:02 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:21:02.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:02 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:21:03.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 128773 4.08 MB/sec execute 50 sec latency 65.013 ms 2024-09-19T06:21:03.691 INFO:tasks.workunit.client.0.smithi053.stdout: 1 412177 13.53 MB/sec execute 47 sec latency 299.648 ms 2024-09-19T06:21:04.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 129597 4.07 MB/sec execute 51 sec latency 12.524 ms 2024-09-19T06:21:04.691 INFO:tasks.workunit.client.0.smithi053.stdout: 1 415726 13.65 MB/sec execute 48 sec latency 33.532 ms 2024-09-19T06:21:04.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:04 smithi053 ceph-mon[27968]: pgmap v347: 129 pgs: 129 active+clean; 69 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 4.7 KiB/s rd, 1.8 MiB/s wr, 79 op/s 2024-09-19T06:21:04.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:04 smithi155 ceph-mon[35292]: pgmap v347: 129 pgs: 129 active+clean; 69 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 4.7 KiB/s rd, 1.8 MiB/s wr, 79 op/s 2024-09-19T06:21:05.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 130349 4.05 MB/sec execute 52 sec latency 19.429 ms 2024-09-19T06:21:05.691 INFO:tasks.workunit.client.0.smithi053.stdout: 1 419269 13.76 MB/sec execute 49 sec latency 41.824 ms 2024-09-19T06:21:06.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 131274 4.15 MB/sec execute 53 sec latency 12.501 ms 2024-09-19T06:21:06.691 INFO:tasks.workunit.client.0.smithi053.stdout: 1 422815 13.87 MB/sec execute 50 sec latency 26.117 ms 2024-09-19T06:21:06.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:06 smithi053 ceph-mon[27968]: pgmap v348: 129 pgs: 129 active+clean; 70 MiB data, 607 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.8 MiB/s wr, 93 op/s 2024-09-19T06:21:06.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:06 smithi155 ceph-mon[35292]: pgmap v348: 129 pgs: 129 active+clean; 70 MiB data, 607 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.8 MiB/s wr, 93 op/s 2024-09-19T06:21:07.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 131855 4.10 MB/sec execute 54 sec latency 11.740 ms 2024-09-19T06:21:07.691 INFO:tasks.workunit.client.0.smithi053.stdout: 1 425326 13.85 MB/sec execute 51 sec latency 80.904 ms 2024-09-19T06:21:08.620 INFO:tasks.workunit.client.1.smithi155.stdout: 1 132664 4.12 MB/sec execute 55 sec latency 14.193 ms 2024-09-19T06:21:08.691 INFO:tasks.workunit.client.0.smithi053.stdout: 1 428153 13.85 MB/sec execute 52 sec latency 116.655 ms 2024-09-19T06:21:08.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:08 smithi053 ceph-mon[27968]: pgmap v349: 129 pgs: 129 active+clean; 69 MiB data, 592 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.6 MiB/s wr, 76 op/s 2024-09-19T06:21:08.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:08 smithi155 ceph-mon[35292]: pgmap v349: 129 pgs: 129 active+clean; 69 MiB data, 592 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.6 MiB/s wr, 76 op/s 2024-09-19T06:21:09.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 133502 4.06 MB/sec execute 56 sec latency 13.913 ms 2024-09-19T06:21:09.691 INFO:tasks.workunit.client.0.smithi053.stdout: 1 431116 13.90 MB/sec execute 53 sec latency 125.099 ms 2024-09-19T06:21:10.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 134539 4.14 MB/sec execute 57 sec latency 17.777 ms 2024-09-19T06:21:10.691 INFO:tasks.workunit.client.0.smithi053.stdout: 1 434406 13.99 MB/sec execute 54 sec latency 33.490 ms 2024-09-19T06:21:10.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:10 smithi053 ceph-mon[27968]: pgmap v350: 129 pgs: 129 active+clean; 70 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 4.8 KiB/s rd, 2.0 MiB/s wr, 98 op/s 2024-09-19T06:21:10.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:10 smithi155 ceph-mon[35292]: pgmap v350: 129 pgs: 129 active+clean; 70 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 4.8 KiB/s rd, 2.0 MiB/s wr, 98 op/s 2024-09-19T06:21:11.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 135094 4.14 MB/sec execute 58 sec latency 12.795 ms 2024-09-19T06:21:11.691 INFO:tasks.workunit.client.0.smithi053.stdout: 1 437306 14.01 MB/sec execute 55 sec latency 33.096 ms 2024-09-19T06:21:12.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 135704 4.09 MB/sec execute 59 sec latency 14.865 ms 2024-09-19T06:21:12.691 INFO:tasks.workunit.client.0.smithi053.stdout: 1 440668 14.09 MB/sec execute 56 sec latency 46.864 ms 2024-09-19T06:21:12.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:12 smithi053 ceph-mon[27968]: pgmap v351: 129 pgs: 129 active+clean; 71 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 85 op/s 2024-09-19T06:21:12.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:12 smithi155 ceph-mon[35292]: pgmap v351: 129 pgs: 129 active+clean; 71 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 85 op/s 2024-09-19T06:21:13.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 136588 4.10 MB/sec execute 60 sec latency 11.302 ms 2024-09-19T06:21:13.691 INFO:tasks.workunit.client.0.smithi053.stdout: 1 444207 14.18 MB/sec execute 57 sec latency 33.364 ms 2024-09-19T06:21:14.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 137387 4.07 MB/sec execute 61 sec latency 12.187 ms 2024-09-19T06:21:14.691 INFO:tasks.workunit.client.0.smithi053.stdout: 1 447691 14.26 MB/sec execute 58 sec latency 33.728 ms 2024-09-19T06:21:14.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:14 smithi053 ceph-mon[27968]: pgmap v352: 129 pgs: 129 active+clean; 71 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 5.1 KiB/s rd, 1.8 MiB/s wr, 93 op/s 2024-09-19T06:21:14.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:14 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:21:14.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:14 smithi155 ceph-mon[35292]: pgmap v352: 129 pgs: 129 active+clean; 71 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 5.1 KiB/s rd, 1.8 MiB/s wr, 93 op/s 2024-09-19T06:21:14.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:14 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:21:15.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 138209 4.11 MB/sec execute 62 sec latency 14.129 ms 2024-09-19T06:21:15.691 INFO:tasks.workunit.client.0.smithi053.stdout: 1 451227 14.35 MB/sec execute 59 sec latency 82.279 ms 2024-09-19T06:21:16.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 138915 4.12 MB/sec execute 63 sec latency 9.919 ms 2024-09-19T06:21:16.691 INFO:tasks.workunit.client.0.smithi053.stdout: 1 454770 14.43 MB/sec execute 60 sec latency 33.194 ms 2024-09-19T06:21:16.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:16 smithi053 ceph-mon[27968]: pgmap v353: 129 pgs: 129 active+clean; 72 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 2.0 MiB/s wr, 105 op/s 2024-09-19T06:21:16.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:16 smithi155 ceph-mon[35292]: pgmap v353: 129 pgs: 129 active+clean; 72 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 2.0 MiB/s wr, 105 op/s 2024-09-19T06:21:17.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 139610 4.09 MB/sec execute 64 sec latency 66.588 ms 2024-09-19T06:21:17.691 INFO:tasks.workunit.client.0.smithi053.stdout: 1 458267 14.50 MB/sec execute 61 sec latency 133.412 ms 2024-09-19T06:21:18.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 140466 4.08 MB/sec execute 65 sec latency 15.285 ms 2024-09-19T06:21:18.691 INFO:tasks.workunit.client.0.smithi053.stdout: 1 460821 14.73 MB/sec execute 62 sec latency 106.098 ms 2024-09-19T06:21:18.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:18 smithi053 ceph-mon[27968]: pgmap v354: 129 pgs: 129 active+clean; 73 MiB data, 605 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.8 MiB/s wr, 85 op/s 2024-09-19T06:21:18.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:18 smithi155 ceph-mon[35292]: pgmap v354: 129 pgs: 129 active+clean; 73 MiB data, 605 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.8 MiB/s wr, 85 op/s 2024-09-19T06:21:19.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 141373 4.13 MB/sec execute 66 sec latency 18.417 ms 2024-09-19T06:21:19.691 INFO:tasks.workunit.client.0.smithi053.stdout: 1 464249 14.79 MB/sec execute 63 sec latency 46.408 ms 2024-09-19T06:21:20.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 142146 4.16 MB/sec execute 67 sec latency 14.881 ms 2024-09-19T06:21:20.691 INFO:tasks.workunit.client.0.smithi053.stdout: 1 467339 14.83 MB/sec execute 64 sec latency 33.100 ms 2024-09-19T06:21:20.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:20 smithi053 ceph-mon[27968]: pgmap v355: 129 pgs: 129 active+clean; 74 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.1 MiB/s wr, 109 op/s 2024-09-19T06:21:20.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:20 smithi155 ceph-mon[35292]: pgmap v355: 129 pgs: 129 active+clean; 74 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.1 MiB/s wr, 109 op/s 2024-09-19T06:21:21.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 142734 4.11 MB/sec execute 68 sec latency 15.363 ms 2024-09-19T06:21:21.691 INFO:tasks.workunit.client.0.smithi053.stdout: 1 470481 14.89 MB/sec execute 65 sec latency 125.086 ms 2024-09-19T06:21:22.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 143545 4.12 MB/sec execute 69 sec latency 12.069 ms 2024-09-19T06:21:22.691 INFO:tasks.workunit.client.0.smithi053.stdout: 1 473466 14.89 MB/sec execute 66 sec latency 55.665 ms 2024-09-19T06:21:22.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:22 smithi053 ceph-mon[27968]: pgmap v356: 129 pgs: 129 active+clean; 74 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.7 MiB/s wr, 83 op/s 2024-09-19T06:21:22.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:22 smithi155 ceph-mon[35292]: pgmap v356: 129 pgs: 129 active+clean; 74 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.7 MiB/s wr, 83 op/s 2024-09-19T06:21:23.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 144344 4.08 MB/sec execute 70 sec latency 13.066 ms 2024-09-19T06:21:23.691 INFO:tasks.workunit.client.0.smithi053.stdout: 1 476900 14.95 MB/sec execute 67 sec latency 44.705 ms 2024-09-19T06:21:24.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 145225 4.13 MB/sec execute 71 sec latency 42.301 ms 2024-09-19T06:21:24.691 INFO:tasks.workunit.client.0.smithi053.stdout: 1 480352 15.01 MB/sec execute 68 sec latency 208.158 ms 2024-09-19T06:21:24.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:24 smithi053 ceph-mon[27968]: pgmap v357: 129 pgs: 129 active+clean; 75 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.0 MiB/s wr, 109 op/s 2024-09-19T06:21:24.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:24 smithi155 ceph-mon[35292]: pgmap v357: 129 pgs: 129 active+clean; 75 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.0 MiB/s wr, 109 op/s 2024-09-19T06:21:25.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 145869 4.14 MB/sec execute 72 sec latency 13.353 ms 2024-09-19T06:21:25.691 INFO:tasks.workunit.client.0.smithi053.stdout: 1 483895 15.07 MB/sec execute 69 sec latency 33.617 ms 2024-09-19T06:21:26.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 146529 4.11 MB/sec execute 73 sec latency 14.948 ms 2024-09-19T06:21:26.691 INFO:tasks.workunit.client.0.smithi053.stdout: 1 486511 15.05 MB/sec execute 70 sec latency 52.679 ms 2024-09-19T06:21:26.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:26 smithi053 ceph-mon[27968]: pgmap v358: 129 pgs: 129 active+clean; 76 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.0 MiB/s wr, 102 op/s 2024-09-19T06:21:26.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:26 smithi155 ceph-mon[35292]: pgmap v358: 129 pgs: 129 active+clean; 76 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.0 MiB/s wr, 102 op/s 2024-09-19T06:21:27.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 147377 4.10 MB/sec execute 74 sec latency 11.862 ms 2024-09-19T06:21:27.692 INFO:tasks.workunit.client.0.smithi053.stdout: 1 489527 15.06 MB/sec execute 71 sec latency 35.505 ms 2024-09-19T06:21:28.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:27 smithi053 ceph-mon[27968]: pgmap v359: 129 pgs: 129 active+clean; 75 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 MiB/s wr, 86 op/s 2024-09-19T06:21:28.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:27 smithi155 ceph-mon[35292]: pgmap v359: 129 pgs: 129 active+clean; 75 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 MiB/s wr, 86 op/s 2024-09-19T06:21:28.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 148329 4.12 MB/sec execute 75 sec latency 18.109 ms 2024-09-19T06:21:28.692 INFO:tasks.workunit.client.0.smithi053.stdout: 1 492655 15.06 MB/sec execute 72 sec latency 42.904 ms 2024-09-19T06:21:29.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:28 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:21:29.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:28 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:21:29.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 149089 4.15 MB/sec execute 76 sec latency 48.141 ms 2024-09-19T06:21:29.692 INFO:tasks.workunit.client.0.smithi053.stdout: 1 494596 15.03 MB/sec execute 73 sec latency 266.825 ms 2024-09-19T06:21:30.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:29 smithi053 ceph-mon[27968]: pgmap v360: 129 pgs: 129 active+clean; 76 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.0 MiB/s wr, 108 op/s 2024-09-19T06:21:30.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:29 smithi155 ceph-mon[35292]: pgmap v360: 129 pgs: 129 active+clean; 76 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.0 MiB/s wr, 108 op/s 2024-09-19T06:21:30.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 149593 4.12 MB/sec execute 77 sec latency 45.986 ms 2024-09-19T06:21:30.692 INFO:tasks.workunit.client.0.smithi053.stdout: 1 494659 14.83 MB/sec execute 74 sec latency 300.094 ms 2024-09-19T06:21:31.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 150288 4.09 MB/sec execute 78 sec latency 46.925 ms 2024-09-19T06:21:31.692 INFO:tasks.workunit.client.0.smithi053.stdout: 1 494735 14.63 MB/sec execute 75 sec latency 308.472 ms 2024-09-19T06:21:32.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:31 smithi155 ceph-mon[35292]: pgmap v361: 129 pgs: 129 active+clean; 76 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.6 MiB/s wr, 80 op/s 2024-09-19T06:21:32.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:31 smithi053 ceph-mon[27968]: pgmap v361: 129 pgs: 129 active+clean; 76 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.6 MiB/s wr, 80 op/s 2024-09-19T06:21:32.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 151140 4.09 MB/sec execute 79 sec latency 49.433 ms 2024-09-19T06:21:32.692 INFO:tasks.workunit.client.0.smithi053.stdout: 1 494779 14.44 MB/sec execute 76 sec latency 325.188 ms 2024-09-19T06:21:33.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 151995 4.12 MB/sec execute 80 sec latency 28.868 ms 2024-09-19T06:21:33.692 INFO:tasks.workunit.client.0.smithi053.stdout: 1 495490 14.31 MB/sec execute 77 sec latency 300.285 ms 2024-09-19T06:21:34.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:33 smithi155 ceph-mon[35292]: pgmap v362: 129 pgs: 129 active+clean; 78 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.0 MiB/s wr, 107 op/s 2024-09-19T06:21:34.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:33 smithi053 ceph-mon[27968]: pgmap v362: 129 pgs: 129 active+clean; 78 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.0 MiB/s wr, 107 op/s 2024-09-19T06:21:34.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 152720 4.14 MB/sec execute 81 sec latency 42.803 ms 2024-09-19T06:21:34.692 INFO:tasks.workunit.client.0.smithi053.stdout: 1 495744 14.13 MB/sec execute 78 sec latency 316.892 ms 2024-09-19T06:21:35.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 153243 4.10 MB/sec execute 82 sec latency 47.486 ms 2024-09-19T06:21:35.692 INFO:tasks.workunit.client.0.smithi053.stdout: 1 496269 13.97 MB/sec execute 79 sec latency 325.469 ms 2024-09-19T06:21:36.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:35 smithi155 ceph-mon[35292]: pgmap v363: 129 pgs: 129 active+clean; 78 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.9 MiB/s wr, 87 op/s 2024-09-19T06:21:36.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:35 smithi053 ceph-mon[27968]: pgmap v363: 129 pgs: 129 active+clean; 78 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.9 MiB/s wr, 87 op/s 2024-09-19T06:21:36.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 153960 4.11 MB/sec execute 83 sec latency 75.113 ms 2024-09-19T06:21:36.692 INFO:tasks.workunit.client.0.smithi053.stdout: 1 496770 13.87 MB/sec execute 80 sec latency 299.445 ms 2024-09-19T06:21:37.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 154726 4.07 MB/sec execute 84 sec latency 96.177 ms 2024-09-19T06:21:37.692 INFO:tasks.workunit.client.0.smithi053.stdout: 1 498389 13.79 MB/sec execute 81 sec latency 323.564 ms 2024-09-19T06:21:38.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:37 smithi053 ceph-mon[27968]: pgmap v364: 129 pgs: 129 active+clean; 77 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.6 MiB/s wr, 83 op/s 2024-09-19T06:21:38.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:37 smithi155 ceph-mon[35292]: pgmap v364: 129 pgs: 129 active+clean; 77 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.6 MiB/s wr, 83 op/s 2024-09-19T06:21:38.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 155743 4.11 MB/sec execute 85 sec latency 11.294 ms 2024-09-19T06:21:38.692 INFO:tasks.workunit.client.0.smithi053.stdout: 1 501754 13.85 MB/sec execute 82 sec latency 41.806 ms 2024-09-19T06:21:39.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 156366 4.13 MB/sec execute 86 sec latency 14.945 ms 2024-09-19T06:21:39.692 INFO:tasks.workunit.client.0.smithi053.stdout: 1 505297 13.91 MB/sec execute 83 sec latency 33.737 ms 2024-09-19T06:21:40.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:39 smithi155 ceph-mon[35292]: pgmap v365: 129 pgs: 129 active+clean; 80 MiB data, 696 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.1 MiB/s wr, 102 op/s 2024-09-19T06:21:40.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:39 smithi053 ceph-mon[27968]: pgmap v365: 129 pgs: 129 active+clean; 80 MiB data, 696 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.1 MiB/s wr, 102 op/s 2024-09-19T06:21:40.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 156960 4.09 MB/sec execute 87 sec latency 16.067 ms 2024-09-19T06:21:40.692 INFO:tasks.workunit.client.0.smithi053.stdout: 1 508774 13.97 MB/sec execute 84 sec latency 42.298 ms 2024-09-19T06:21:41.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 157754 4.10 MB/sec execute 88 sec latency 12.144 ms 2024-09-19T06:21:41.692 INFO:tasks.workunit.client.0.smithi053.stdout: 1 512315 14.03 MB/sec execute 85 sec latency 33.762 ms 2024-09-19T06:21:42.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:41 smithi155 ceph-mon[35292]: pgmap v366: 129 pgs: 129 active+clean; 79 MiB data, 696 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.6 MiB/s wr, 77 op/s 2024-09-19T06:21:42.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:41 smithi053 ceph-mon[27968]: pgmap v366: 129 pgs: 129 active+clean; 79 MiB data, 696 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.6 MiB/s wr, 77 op/s 2024-09-19T06:21:42.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 158535 4.07 MB/sec execute 89 sec latency 13.379 ms 2024-09-19T06:21:42.692 INFO:tasks.workunit.client.0.smithi053.stdout: 1 514561 14.01 MB/sec execute 86 sec latency 236.660 ms 2024-09-19T06:21:43.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 159417 4.11 MB/sec execute 90 sec latency 11.698 ms 2024-09-19T06:21:43.692 INFO:tasks.workunit.client.0.smithi053.stdout: 1 517602 14.01 MB/sec execute 87 sec latency 44.750 ms 2024-09-19T06:21:44.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:43 smithi155 ceph-mon[35292]: pgmap v367: 129 pgs: 129 active+clean; 80 MiB data, 702 MiB used, 715 GiB / 715 GiB avail; 5.4 KiB/s rd, 1.9 MiB/s wr, 102 op/s 2024-09-19T06:21:44.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:43 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:21:44.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:43 smithi053 ceph-mon[27968]: pgmap v367: 129 pgs: 129 active+clean; 80 MiB data, 702 MiB used, 715 GiB / 715 GiB avail; 5.4 KiB/s rd, 1.9 MiB/s wr, 102 op/s 2024-09-19T06:21:44.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:43 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:21:44.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 160075 4.11 MB/sec execute 91 sec latency 12.766 ms 2024-09-19T06:21:44.692 INFO:tasks.workunit.client.0.smithi053.stdout: 1 521051 14.06 MB/sec execute 88 sec latency 41.274 ms 2024-09-19T06:21:45.621 INFO:tasks.workunit.client.1.smithi155.stdout: 1 160775 4.09 MB/sec execute 92 sec latency 16.689 ms 2024-09-19T06:21:45.693 INFO:tasks.workunit.client.0.smithi053.stdout: 1 524106 14.10 MB/sec execute 89 sec latency 33.175 ms 2024-09-19T06:21:46.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:45 smithi155 ceph-mon[35292]: pgmap v368: 129 pgs: 129 active+clean; 79 MiB data, 706 MiB used, 715 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.8 MiB/s wr, 82 op/s 2024-09-19T06:21:46.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:45 smithi053 ceph-mon[27968]: pgmap v368: 129 pgs: 129 active+clean; 79 MiB data, 706 MiB used, 715 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.8 MiB/s wr, 82 op/s 2024-09-19T06:21:46.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 161642 4.09 MB/sec execute 93 sec latency 16.929 ms 2024-09-19T06:21:46.692 INFO:tasks.workunit.client.0.smithi053.stdout: 1 527252 14.15 MB/sec execute 90 sec latency 33.504 ms 2024-09-19T06:21:47.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 162392 4.10 MB/sec execute 94 sec latency 60.782 ms 2024-09-19T06:21:47.692 INFO:tasks.workunit.client.0.smithi053.stdout: 1 530233 14.16 MB/sec execute 91 sec latency 41.461 ms 2024-09-19T06:21:48.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:47 smithi155 ceph-mon[35292]: pgmap v369: 129 pgs: 129 active+clean; 81 MiB data, 709 MiB used, 715 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.7 MiB/s wr, 95 op/s 2024-09-19T06:21:48.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:47 smithi053 ceph-mon[27968]: pgmap v369: 129 pgs: 129 active+clean; 81 MiB data, 709 MiB used, 715 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.7 MiB/s wr, 95 op/s 2024-09-19T06:21:48.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 163195 4.12 MB/sec execute 95 sec latency 12.750 ms 2024-09-19T06:21:48.693 INFO:tasks.workunit.client.0.smithi053.stdout: 1 533697 14.21 MB/sec execute 92 sec latency 33.288 ms 2024-09-19T06:21:49.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 163777 4.10 MB/sec execute 96 sec latency 12.299 ms 2024-09-19T06:21:49.692 INFO:tasks.workunit.client.0.smithi053.stdout: 1 537231 14.27 MB/sec execute 93 sec latency 27.142 ms 2024-09-19T06:21:50.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:49 smithi155 ceph-mon[35292]: pgmap v370: 129 pgs: 129 active+clean; 81 MiB data, 696 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.1 MiB/s wr, 102 op/s 2024-09-19T06:21:50.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:49 smithi053 ceph-mon[27968]: pgmap v370: 129 pgs: 129 active+clean; 81 MiB data, 696 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.1 MiB/s wr, 102 op/s 2024-09-19T06:21:50.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 164607 4.11 MB/sec execute 97 sec latency 14.215 ms 2024-09-19T06:21:50.693 INFO:tasks.workunit.client.0.smithi053.stdout: 1 540668 14.32 MB/sec execute 94 sec latency 83.114 ms 2024-09-19T06:21:51.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 165491 4.08 MB/sec execute 98 sec latency 11.931 ms 2024-09-19T06:21:51.693 INFO:tasks.workunit.client.0.smithi053.stdout: 1 544199 14.37 MB/sec execute 95 sec latency 46.749 ms 2024-09-19T06:21:52.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:51 smithi155 ceph-mon[35292]: pgmap v371: 129 pgs: 129 active+clean; 81 MiB data, 695 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.5 MiB/s wr, 79 op/s 2024-09-19T06:21:52.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:51 smithi053 ceph-mon[27968]: pgmap v371: 129 pgs: 129 active+clean; 81 MiB data, 695 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.5 MiB/s wr, 79 op/s 2024-09-19T06:21:52.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 166466 4.12 MB/sec execute 99 sec latency 11.595 ms 2024-09-19T06:21:52.692 INFO:tasks.workunit.client.0.smithi053.stdout: 1 546824 14.36 MB/sec execute 96 sec latency 33.302 ms 2024-09-19T06:21:53.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 167047 4.12 MB/sec execute 100 sec latency 13.302 ms 2024-09-19T06:21:53.693 INFO:tasks.workunit.client.0.smithi053.stdout: 1 549549 14.34 MB/sec execute 97 sec latency 91.795 ms 2024-09-19T06:21:54.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:53 smithi155 ceph-mon[35292]: pgmap v372: 129 pgs: 129 active+clean; 84 MiB data, 704 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.1 MiB/s wr, 105 op/s 2024-09-19T06:21:54.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:53 smithi053 ceph-mon[27968]: pgmap v372: 129 pgs: 129 active+clean; 84 MiB data, 704 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.1 MiB/s wr, 105 op/s 2024-09-19T06:21:54.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 167676 4.09 MB/sec execute 101 sec latency 18.857 ms 2024-09-19T06:21:54.693 INFO:tasks.workunit.client.0.smithi053.stdout: 1 552253 14.37 MB/sec execute 98 sec latency 91.692 ms 2024-09-19T06:21:55.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 168575 4.10 MB/sec execute 102 sec latency 10.218 ms 2024-09-19T06:21:55.693 INFO:tasks.workunit.client.0.smithi053.stdout: 1 555150 14.37 MB/sec execute 99 sec latency 40.273 ms 2024-09-19T06:21:56.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:55 smithi155 ceph-mon[35292]: pgmap v373: 129 pgs: 129 active+clean; 83 MiB data, 705 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.0 MiB/s wr, 87 op/s 2024-09-19T06:21:56.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:55 smithi053 ceph-mon[27968]: pgmap v373: 129 pgs: 129 active+clean; 83 MiB data, 705 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.0 MiB/s wr, 87 op/s 2024-09-19T06:21:56.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 169347 4.08 MB/sec execute 103 sec latency 11.912 ms 2024-09-19T06:21:56.693 INFO:tasks.workunit.client.0.smithi053.stdout: 1 558518 14.42 MB/sec execute 100 sec latency 33.514 ms 2024-09-19T06:21:57.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 170264 4.14 MB/sec execute 104 sec latency 12.383 ms 2024-09-19T06:21:57.693 INFO:tasks.workunit.client.0.smithi053.stdout: 1 561942 14.46 MB/sec execute 101 sec latency 191.848 ms 2024-09-19T06:21:58.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:57 smithi155 ceph-mon[35292]: pgmap v374: 129 pgs: 129 active+clean; 84 MiB data, 689 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.9 MiB/s wr, 96 op/s 2024-09-19T06:21:58.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:57 smithi053 ceph-mon[27968]: pgmap v374: 129 pgs: 129 active+clean; 84 MiB data, 689 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.9 MiB/s wr, 96 op/s 2024-09-19T06:21:58.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 170870 4.11 MB/sec execute 105 sec latency 9.740 ms 2024-09-19T06:21:58.693 INFO:tasks.workunit.client.0.smithi053.stdout: 1 564579 14.46 MB/sec execute 102 sec latency 158.646 ms 2024-09-19T06:21:59.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:58 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:21:59.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:58 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:21:59.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 171677 4.12 MB/sec execute 106 sec latency 15.291 ms 2024-09-19T06:21:59.693 INFO:tasks.workunit.client.0.smithi053.stdout: 1 567794 14.48 MB/sec execute 103 sec latency 33.249 ms 2024-09-19T06:22:00.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:21:59 smithi155 ceph-mon[35292]: pgmap v375: 129 pgs: 129 active+clean; 84 MiB data, 683 MiB used, 715 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.1 MiB/s wr, 94 op/s 2024-09-19T06:22:00.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:21:59 smithi053 ceph-mon[27968]: pgmap v375: 129 pgs: 129 active+clean; 84 MiB data, 683 MiB used, 715 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.1 MiB/s wr, 94 op/s 2024-09-19T06:22:00.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 172568 4.10 MB/sec execute 107 sec latency 15.413 ms 2024-09-19T06:22:00.693 INFO:tasks.workunit.client.0.smithi053.stdout: 1 570761 14.48 MB/sec execute 104 sec latency 33.985 ms 2024-09-19T06:22:01.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 173460 4.12 MB/sec execute 108 sec latency 12.810 ms 2024-09-19T06:22:01.693 INFO:tasks.workunit.client.0.smithi053.stdout: 1 573778 14.50 MB/sec execute 105 sec latency 44.431 ms 2024-09-19T06:22:02.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:01 smithi155 ceph-mon[35292]: pgmap v376: 129 pgs: 129 active+clean; 84 MiB data, 683 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.7 MiB/s wr, 83 op/s 2024-09-19T06:22:02.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:01 smithi053 ceph-mon[27968]: pgmap v376: 129 pgs: 129 active+clean; 84 MiB data, 683 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.7 MiB/s wr, 83 op/s 2024-09-19T06:22:02.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 174097 4.13 MB/sec execute 109 sec latency 11.994 ms 2024-09-19T06:22:02.693 INFO:tasks.workunit.client.0.smithi053.stdout: 1 576423 14.51 MB/sec execute 106 sec latency 40.020 ms 2024-09-19T06:22:03.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:02 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:22:03.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:02 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:22:03.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 174603 4.10 MB/sec execute 110 sec latency 39.024 ms 2024-09-19T06:22:03.693 INFO:tasks.workunit.client.0.smithi053.stdout: 1 578737 14.50 MB/sec execute 107 sec latency 299.596 ms 2024-09-19T06:22:04.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:03 smithi155 ceph-mon[35292]: pgmap v377: 129 pgs: 129 active+clean; 84 MiB data, 686 MiB used, 715 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.1 MiB/s wr, 111 op/s 2024-09-19T06:22:04.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:03 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:22:04.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:03 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:22:04.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:03 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:22:04.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:03 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:22:04.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:03 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:22:04.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:03 smithi053 ceph-mon[27968]: pgmap v377: 129 pgs: 129 active+clean; 84 MiB data, 686 MiB used, 715 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.1 MiB/s wr, 111 op/s 2024-09-19T06:22:04.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:03 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:22:04.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:03 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:22:04.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:03 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:22:04.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:03 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:22:04.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:03 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:22:04.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 175295 4.11 MB/sec execute 111 sec latency 99.069 ms 2024-09-19T06:22:04.693 INFO:tasks.workunit.client.0.smithi053.stdout: 1 579654 14.41 MB/sec execute 108 sec latency 366.769 ms 2024-09-19T06:22:05.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 176130 4.09 MB/sec execute 112 sec latency 38.544 ms 2024-09-19T06:22:05.693 INFO:tasks.workunit.client.0.smithi053.stdout: 1 579663 14.27 MB/sec execute 109 sec latency 291.785 ms 2024-09-19T06:22:06.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:05 smithi155 ceph-mon[35292]: pgmap v378: 129 pgs: 129 active+clean; 86 MiB data, 679 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.9 MiB/s wr, 90 op/s 2024-09-19T06:22:06.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:05 smithi053 ceph-mon[27968]: pgmap v378: 129 pgs: 129 active+clean; 86 MiB data, 679 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.9 MiB/s wr, 90 op/s 2024-09-19T06:22:06.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 177031 4.11 MB/sec execute 113 sec latency 64.789 ms 2024-09-19T06:22:06.693 INFO:tasks.workunit.client.0.smithi053.stdout: 1 579715 14.15 MB/sec execute 110 sec latency 308.523 ms 2024-09-19T06:22:07.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 177639 4.12 MB/sec execute 114 sec latency 38.952 ms 2024-09-19T06:22:07.693 INFO:tasks.workunit.client.0.smithi053.stdout: 1 579722 14.02 MB/sec execute 111 sec latency 291.872 ms 2024-09-19T06:22:08.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:07 smithi155 ceph-mon[35292]: pgmap v379: 129 pgs: 129 active+clean; 86 MiB data, 683 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.8 MiB/s wr, 88 op/s 2024-09-19T06:22:08.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:07 smithi053 ceph-mon[27968]: pgmap v379: 129 pgs: 129 active+clean; 86 MiB data, 683 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.8 MiB/s wr, 88 op/s 2024-09-19T06:22:08.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 178238 4.09 MB/sec execute 115 sec latency 31.784 ms 2024-09-19T06:22:08.693 INFO:tasks.workunit.client.0.smithi053.stdout: 1 579785 13.90 MB/sec execute 112 sec latency 291.836 ms 2024-09-19T06:22:09.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 179016 4.10 MB/sec execute 116 sec latency 13.068 ms 2024-09-19T06:22:09.693 INFO:tasks.workunit.client.0.smithi053.stdout: 1 579864 13.77 MB/sec execute 113 sec latency 308.291 ms 2024-09-19T06:22:10.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:09 smithi155 ceph-mon[35292]: pgmap v380: 129 pgs: 129 active+clean; 88 MiB data, 688 MiB used, 715 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.1 MiB/s wr, 96 op/s 2024-09-19T06:22:10.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:09 smithi053 ceph-mon[27968]: pgmap v380: 129 pgs: 129 active+clean; 88 MiB data, 688 MiB used, 715 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.1 MiB/s wr, 96 op/s 2024-09-19T06:22:10.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 179796 4.08 MB/sec execute 117 sec latency 40.300 ms 2024-09-19T06:22:10.693 INFO:tasks.workunit.client.0.smithi053.stdout: 1 579970 13.66 MB/sec execute 114 sec latency 316.770 ms 2024-09-19T06:22:11.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 180686 4.11 MB/sec execute 118 sec latency 12.221 ms 2024-09-19T06:22:11.693 INFO:tasks.workunit.client.0.smithi053.stdout: 1 583450 13.70 MB/sec execute 115 sec latency 33.595 ms 2024-09-19T06:22:12.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:11 smithi053 ceph-mon[27968]: pgmap v381: 129 pgs: 129 active+clean; 87 MiB data, 692 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.6 MiB/s wr, 80 op/s 2024-09-19T06:22:12.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:11 smithi155 ceph-mon[35292]: pgmap v381: 129 pgs: 129 active+clean; 87 MiB data, 692 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.6 MiB/s wr, 80 op/s 2024-09-19T06:22:12.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 181340 4.11 MB/sec execute 119 sec latency 12.126 ms 2024-09-19T06:22:12.693 INFO:tasks.workunit.client.0.smithi053.stdout: 1 586879 13.75 MB/sec execute 116 sec latency 43.106 ms 2024-09-19T06:22:13.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 182054 4.09 MB/sec execute 120 sec latency 16.195 ms 2024-09-19T06:22:13.694 INFO:tasks.workunit.client.0.smithi053.stdout: 1 590353 13.79 MB/sec execute 117 sec latency 162.177 ms 2024-09-19T06:22:14.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:13 smithi155 ceph-mon[35292]: pgmap v382: 129 pgs: 129 active+clean; 89 MiB data, 717 MiB used, 715 GiB / 715 GiB avail; 4.8 KiB/s rd, 2.0 MiB/s wr, 100 op/s 2024-09-19T06:22:14.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:13 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:22:14.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:13 smithi053 ceph-mon[27968]: pgmap v382: 129 pgs: 129 active+clean; 89 MiB data, 717 MiB used, 715 GiB / 715 GiB avail; 4.8 KiB/s rd, 2.0 MiB/s wr, 100 op/s 2024-09-19T06:22:14.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:13 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:22:14.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 182932 4.09 MB/sec execute 121 sec latency 16.689 ms 2024-09-19T06:22:14.694 INFO:tasks.workunit.client.0.smithi053.stdout: 1 593846 13.83 MB/sec execute 118 sec latency 36.034 ms 2024-09-19T06:22:15.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 183807 4.11 MB/sec execute 122 sec latency 21.569 ms 2024-09-19T06:22:15.694 INFO:tasks.workunit.client.0.smithi053.stdout: 1 596459 13.84 MB/sec execute 119 sec latency 66.778 ms 2024-09-19T06:22:16.235 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:15 smithi155 ceph-mon[35292]: pgmap v383: 129 pgs: 129 active+clean; 89 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.9 MiB/s wr, 92 op/s 2024-09-19T06:22:16.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:15 smithi053 ceph-mon[27968]: pgmap v383: 129 pgs: 129 active+clean; 89 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.9 MiB/s wr, 92 op/s 2024-09-19T06:22:16.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 184575 4.12 MB/sec execute 123 sec latency 12.213 ms 2024-09-19T06:22:16.694 INFO:tasks.workunit.client.0.smithi053.stdout: 1 599632 13.87 MB/sec execute 120 sec latency 35.916 ms 2024-09-19T06:22:17.623 INFO:tasks.workunit.client.1.smithi155.stdout: 1 185082 4.10 MB/sec execute 124 sec latency 15.852 ms 2024-09-19T06:22:17.694 INFO:tasks.workunit.client.0.smithi053.stdout: 1 602601 13.86 MB/sec execute 121 sec latency 41.925 ms 2024-09-19T06:22:18.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:17 smithi155 ceph-mon[35292]: pgmap v384: 129 pgs: 129 active+clean; 90 MiB data, 723 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.7 MiB/s wr, 89 op/s 2024-09-19T06:22:18.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:17 smithi053 ceph-mon[27968]: pgmap v384: 129 pgs: 129 active+clean; 90 MiB data, 723 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.7 MiB/s wr, 89 op/s 2024-09-19T06:22:18.622 INFO:tasks.workunit.client.1.smithi155.stdout: 1 185911 4.11 MB/sec execute 125 sec latency 14.805 ms 2024-09-19T06:22:18.694 INFO:tasks.workunit.client.0.smithi053.stdout: 1 605694 13.89 MB/sec execute 122 sec latency 25.330 ms 2024-09-19T06:22:19.623 INFO:tasks.workunit.client.1.smithi155.stdout: 1 186781 4.09 MB/sec execute 126 sec latency 12.037 ms 2024-09-19T06:22:19.694 INFO:tasks.workunit.client.0.smithi053.stdout: 1 608334 13.90 MB/sec execute 123 sec latency 49.510 ms 2024-09-19T06:22:20.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:19 smithi155 ceph-mon[35292]: pgmap v385: 129 pgs: 129 active+clean; 89 MiB data, 724 MiB used, 715 GiB / 715 GiB avail; 4.8 KiB/s rd, 2.0 MiB/s wr, 96 op/s 2024-09-19T06:22:20.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:19 smithi053 ceph-mon[27968]: pgmap v385: 129 pgs: 129 active+clean; 89 MiB data, 724 MiB used, 715 GiB / 715 GiB avail; 4.8 KiB/s rd, 2.0 MiB/s wr, 96 op/s 2024-09-19T06:22:20.623 INFO:tasks.workunit.client.1.smithi155.stdout: 1 187739 4.12 MB/sec execute 127 sec latency 13.802 ms 2024-09-19T06:22:20.694 INFO:tasks.workunit.client.0.smithi053.stdout: 1 611778 13.94 MB/sec execute 124 sec latency 41.784 ms 2024-09-19T06:22:21.623 INFO:tasks.workunit.client.1.smithi155.stdout: 1 188323 4.12 MB/sec execute 128 sec latency 11.956 ms 2024-09-19T06:22:21.694 INFO:tasks.workunit.client.0.smithi053.stdout: 1 615242 13.98 MB/sec execute 125 sec latency 83.233 ms 2024-09-19T06:22:22.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:21 smithi155 ceph-mon[35292]: pgmap v386: 129 pgs: 129 active+clean; 91 MiB data, 726 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.6 MiB/s wr, 86 op/s 2024-09-19T06:22:22.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:21 smithi053 ceph-mon[27968]: pgmap v386: 129 pgs: 129 active+clean; 91 MiB data, 726 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.6 MiB/s wr, 86 op/s 2024-09-19T06:22:22.623 INFO:tasks.workunit.client.1.smithi155.stdout: 1 188950 4.09 MB/sec execute 129 sec latency 16.090 ms 2024-09-19T06:22:22.694 INFO:tasks.workunit.client.0.smithi053.stdout: 1 618723 14.02 MB/sec execute 126 sec latency 50.116 ms 2024-09-19T06:22:23.623 INFO:tasks.workunit.client.1.smithi155.stdout: 1 189791 4.10 MB/sec execute 130 sec latency 48.451 ms 2024-09-19T06:22:23.694 INFO:tasks.workunit.client.0.smithi053.stdout: 1 622262 14.06 MB/sec execute 127 sec latency 25.220 ms 2024-09-19T06:22:24.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:23 smithi155 ceph-mon[35292]: pgmap v387: 129 pgs: 129 active+clean; 90 MiB data, 730 MiB used, 715 GiB / 715 GiB avail; 4.8 KiB/s rd, 1.8 MiB/s wr, 94 op/s 2024-09-19T06:22:24.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:23 smithi053 ceph-mon[27968]: pgmap v387: 129 pgs: 129 active+clean; 90 MiB data, 730 MiB used, 715 GiB / 715 GiB avail; 4.8 KiB/s rd, 1.8 MiB/s wr, 94 op/s 2024-09-19T06:22:24.623 INFO:tasks.workunit.client.1.smithi155.stdout: 1 190560 4.09 MB/sec execute 131 sec latency 16.782 ms 2024-09-19T06:22:24.694 INFO:tasks.workunit.client.0.smithi053.stdout: 1 625758 14.10 MB/sec execute 128 sec latency 46.588 ms 2024-09-19T06:22:25.623 INFO:tasks.workunit.client.1.smithi155.stdout: 1 191333 4.10 MB/sec execute 132 sec latency 14.578 ms 2024-09-19T06:22:25.694 INFO:tasks.workunit.client.0.smithi053.stdout: 1 627842 14.07 MB/sec execute 129 sec latency 133.552 ms 2024-09-19T06:22:26.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:25 smithi155 ceph-mon[35292]: pgmap v388: 129 pgs: 129 active+clean; 92 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.9 MiB/s wr, 100 op/s 2024-09-19T06:22:26.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:25 smithi053 ceph-mon[27968]: pgmap v388: 129 pgs: 129 active+clean; 92 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.9 MiB/s wr, 100 op/s 2024-09-19T06:22:26.623 INFO:tasks.workunit.client.1.smithi155.stdout: 1 192008 4.11 MB/sec execute 133 sec latency 11.180 ms 2024-09-19T06:22:26.694 INFO:tasks.workunit.client.0.smithi053.stdout: 1 630509 14.07 MB/sec execute 130 sec latency 46.748 ms 2024-09-19T06:22:27.623 INFO:tasks.workunit.client.1.smithi155.stdout: 1 192722 4.09 MB/sec execute 134 sec latency 16.056 ms 2024-09-19T06:22:27.694 INFO:tasks.workunit.client.0.smithi053.stdout: 1 633156 14.07 MB/sec execute 131 sec latency 33.631 ms 2024-09-19T06:22:28.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:27 smithi155 ceph-mon[35292]: pgmap v389: 129 pgs: 129 active+clean; 92 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.7 MiB/s wr, 84 op/s 2024-09-19T06:22:28.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:27 smithi053 ceph-mon[27968]: pgmap v389: 129 pgs: 129 active+clean; 92 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.7 MiB/s wr, 84 op/s 2024-09-19T06:22:28.623 INFO:tasks.workunit.client.1.smithi155.stdout: 1 193571 4.09 MB/sec execute 135 sec latency 16.196 ms 2024-09-19T06:22:28.694 INFO:tasks.workunit.client.0.smithi053.stdout: 1 636596 14.11 MB/sec execute 132 sec latency 34.820 ms 2024-09-19T06:22:29.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:28 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:22:29.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:28 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:22:29.623 INFO:tasks.workunit.client.1.smithi155.stdout: 1 194420 4.10 MB/sec execute 136 sec latency 13.048 ms 2024-09-19T06:22:29.694 INFO:tasks.workunit.client.0.smithi053.stdout: 1 640064 14.15 MB/sec execute 133 sec latency 55.020 ms 2024-09-19T06:22:30.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:29 smithi155 ceph-mon[35292]: pgmap v390: 129 pgs: 129 active+clean; 92 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.0 MiB/s wr, 99 op/s 2024-09-19T06:22:30.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:29 smithi053 ceph-mon[27968]: pgmap v390: 129 pgs: 129 active+clean; 92 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.0 MiB/s wr, 99 op/s 2024-09-19T06:22:30.623 INFO:tasks.workunit.client.1.smithi155.stdout: 1 195180 4.12 MB/sec execute 137 sec latency 14.719 ms 2024-09-19T06:22:30.694 INFO:tasks.workunit.client.0.smithi053.stdout: 1 643532 14.18 MB/sec execute 134 sec latency 100.105 ms 2024-09-19T06:22:31.623 INFO:tasks.workunit.client.1.smithi155.stdout: 1 195713 4.10 MB/sec execute 138 sec latency 25.026 ms 2024-09-19T06:22:31.695 INFO:tasks.workunit.client.0.smithi053.stdout: 1 645566 14.15 MB/sec execute 135 sec latency 225.960 ms 2024-09-19T06:22:32.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:31 smithi155 ceph-mon[35292]: pgmap v391: 129 pgs: 129 active+clean; 92 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.7 MiB/s wr, 92 op/s 2024-09-19T06:22:32.281 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:31 smithi053 ceph-mon[27968]: pgmap v391: 129 pgs: 129 active+clean; 92 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.7 MiB/s wr, 92 op/s 2024-09-19T06:22:32.623 INFO:tasks.workunit.client.1.smithi155.stdout: 1 196506 4.11 MB/sec execute 139 sec latency 16.461 ms 2024-09-19T06:22:32.694 INFO:tasks.workunit.client.0.smithi053.stdout: 1 648697 14.16 MB/sec execute 136 sec latency 33.882 ms 2024-09-19T06:22:33.623 INFO:tasks.workunit.client.1.smithi155.stdout: 1 197281 4.08 MB/sec execute 140 sec latency 16.477 ms 2024-09-19T06:22:33.694 INFO:tasks.workunit.client.0.smithi053.stdout: 1 651529 14.19 MB/sec execute 137 sec latency 59.566 ms 2024-09-19T06:22:34.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:33 smithi155 ceph-mon[35292]: pgmap v392: 129 pgs: 129 active+clean; 92 MiB data, 770 MiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 1.9 MiB/s wr, 102 op/s 2024-09-19T06:22:34.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:33 smithi053 ceph-mon[27968]: pgmap v392: 129 pgs: 129 active+clean; 92 MiB data, 770 MiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 1.9 MiB/s wr, 102 op/s 2024-09-19T06:22:34.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 198261 4.10 MB/sec execute 141 sec latency 19.225 ms 2024-09-19T06:22:34.694 INFO:tasks.workunit.client.0.smithi053.stdout: 1 654367 14.19 MB/sec execute 138 sec latency 33.131 ms 2024-09-19T06:22:35.623 INFO:tasks.workunit.client.1.smithi155.stdout: 1 198882 4.12 MB/sec execute 142 sec latency 11.963 ms 2024-09-19T06:22:35.694 INFO:tasks.workunit.client.0.smithi053.stdout: 1 657795 14.22 MB/sec execute 139 sec latency 27.166 ms 2024-09-19T06:22:36.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:35 smithi053 ceph-mon[27968]: pgmap v393: 129 pgs: 129 active+clean; 94 MiB data, 790 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.0 MiB/s wr, 98 op/s 2024-09-19T06:22:36.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:35 smithi155 ceph-mon[35292]: pgmap v393: 129 pgs: 129 active+clean; 94 MiB data, 790 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.0 MiB/s wr, 98 op/s 2024-09-19T06:22:36.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 199462 4.09 MB/sec execute 143 sec latency 15.476 ms 2024-09-19T06:22:36.791 INFO:tasks.workunit.client.0.smithi053.stdout: 1 661330 14.26 MB/sec execute 140 sec latency 41.767 ms 2024-09-19T06:22:37.623 INFO:tasks.workunit.client.1.smithi155.stdout: 1 200229 4.10 MB/sec execute 144 sec latency 28.419 ms 2024-09-19T06:22:37.695 INFO:tasks.workunit.client.0.smithi053.stdout: 1 661519 14.16 MB/sec execute 141 sec latency 283.444 ms 2024-09-19T06:22:38.284 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:37 smithi155 ceph-mon[35292]: pgmap v394: 129 pgs: 129 active+clean; 94 MiB data, 795 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 77 op/s 2024-09-19T06:22:38.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:37 smithi053 ceph-mon[27968]: pgmap v394: 129 pgs: 129 active+clean; 94 MiB data, 795 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 77 op/s 2024-09-19T06:22:38.623 INFO:tasks.workunit.client.1.smithi155.stdout: 1 201002 4.08 MB/sec execute 145 sec latency 78.924 ms 2024-09-19T06:22:38.695 INFO:tasks.workunit.client.0.smithi053.stdout: 1 662170 14.09 MB/sec execute 142 sec latency 275.186 ms 2024-09-19T06:22:39.623 INFO:tasks.workunit.client.1.smithi155.stdout: 1 201926 4.10 MB/sec execute 146 sec latency 46.091 ms 2024-09-19T06:22:39.695 INFO:tasks.workunit.client.0.smithi053.stdout: 1 662424 13.99 MB/sec execute 143 sec latency 308.378 ms 2024-09-19T06:22:40.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:39 smithi053 ceph-mon[27968]: pgmap v395: 129 pgs: 129 active+clean; 94 MiB data, 800 MiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.0 MiB/s wr, 99 op/s 2024-09-19T06:22:40.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:39 smithi155 ceph-mon[35292]: pgmap v395: 129 pgs: 129 active+clean; 94 MiB data, 800 MiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.0 MiB/s wr, 99 op/s 2024-09-19T06:22:40.623 INFO:tasks.workunit.client.1.smithi155.stdout: 1 202503 4.11 MB/sec execute 147 sec latency 46.635 ms 2024-09-19T06:22:40.695 INFO:tasks.workunit.client.0.smithi053.stdout: 1 663010 13.91 MB/sec execute 144 sec latency 289.397 ms 2024-09-19T06:22:41.623 INFO:tasks.workunit.client.1.smithi155.stdout: 1 203098 4.08 MB/sec execute 148 sec latency 43.212 ms 2024-09-19T06:22:41.695 INFO:tasks.workunit.client.0.smithi053.stdout: 1 663511 13.85 MB/sec execute 145 sec latency 324.081 ms 2024-09-19T06:22:42.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:41 smithi053 ceph-mon[27968]: pgmap v396: 129 pgs: 129 active+clean; 94 MiB data, 800 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.6 MiB/s wr, 82 op/s 2024-09-19T06:22:42.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:41 smithi155 ceph-mon[35292]: pgmap v396: 129 pgs: 129 active+clean; 94 MiB data, 800 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.6 MiB/s wr, 82 op/s 2024-09-19T06:22:42.623 INFO:tasks.workunit.client.1.smithi155.stdout: 1 203835 4.09 MB/sec execute 149 sec latency 87.327 ms 2024-09-19T06:22:42.695 INFO:tasks.workunit.client.0.smithi053.stdout: 1 663833 13.77 MB/sec execute 146 sec latency 299.142 ms 2024-09-19T06:22:43.623 INFO:tasks.workunit.client.1.smithi155.stdout: 1 204635 4.07 MB/sec execute 150 sec latency 41.986 ms 2024-09-19T06:22:43.695 INFO:tasks.workunit.client.0.smithi053.stdout: 1 664750 13.71 MB/sec execute 147 sec latency 283.135 ms 2024-09-19T06:22:44.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:43 smithi053 ceph-mon[27968]: pgmap v397: 129 pgs: 129 active+clean; 97 MiB data, 809 MiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.0 MiB/s wr, 105 op/s 2024-09-19T06:22:44.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:43 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:22:44.484 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:43 smithi155 ceph-mon[35292]: pgmap v397: 129 pgs: 129 active+clean; 97 MiB data, 809 MiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.0 MiB/s wr, 105 op/s 2024-09-19T06:22:44.485 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:43 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:22:44.623 INFO:tasks.workunit.client.1.smithi155.stdout: 1 205518 4.09 MB/sec execute 151 sec latency 11.629 ms 2024-09-19T06:22:44.695 INFO:tasks.workunit.client.0.smithi053.stdout: 1 664881 13.62 MB/sec execute 148 sec latency 577.999 ms 2024-09-19T06:22:45.623 INFO:tasks.workunit.client.1.smithi155.stdout: 1 206198 4.10 MB/sec execute 152 sec latency 12.012 ms 2024-09-19T06:22:45.695 INFO:tasks.workunit.client.0.smithi053.stdout: 1 668364 13.65 MB/sec execute 149 sec latency 64.172 ms 2024-09-19T06:22:46.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:45 smithi053 ceph-mon[27968]: pgmap v398: 129 pgs: 129 active+clean; 97 MiB data, 836 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.9 MiB/s wr, 84 op/s 2024-09-19T06:22:46.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:45 smithi155 ceph-mon[35292]: pgmap v398: 129 pgs: 129 active+clean; 97 MiB data, 836 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.9 MiB/s wr, 84 op/s 2024-09-19T06:22:46.623 INFO:tasks.workunit.client.1.smithi155.stdout: 1 206905 4.08 MB/sec execute 153 sec latency 15.154 ms 2024-09-19T06:22:46.695 INFO:tasks.workunit.client.0.smithi053.stdout: 1 671851 13.69 MB/sec execute 150 sec latency 108.589 ms 2024-09-19T06:22:47.623 INFO:tasks.workunit.client.1.smithi155.stdout: 1 207783 4.08 MB/sec execute 154 sec latency 16.083 ms 2024-09-19T06:22:47.695 INFO:tasks.workunit.client.0.smithi053.stdout: 1 675450 13.72 MB/sec execute 151 sec latency 39.191 ms 2024-09-19T06:22:48.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:47 smithi053 ceph-mon[27968]: pgmap v399: 129 pgs: 129 active+clean; 97 MiB data, 836 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.6 MiB/s wr, 81 op/s 2024-09-19T06:22:48.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:47 smithi155 ceph-mon[35292]: pgmap v399: 129 pgs: 129 active+clean; 97 MiB data, 836 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.6 MiB/s wr, 81 op/s 2024-09-19T06:22:48.623 INFO:tasks.workunit.client.1.smithi155.stdout: 1 208691 4.10 MB/sec execute 155 sec latency 13.599 ms 2024-09-19T06:22:48.695 INFO:tasks.workunit.client.0.smithi053.stdout: 1 678988 13.76 MB/sec execute 152 sec latency 33.753 ms 2024-09-19T06:22:49.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 209486 4.11 MB/sec execute 156 sec latency 10.361 ms 2024-09-19T06:22:49.695 INFO:tasks.workunit.client.0.smithi053.stdout: 1 681611 13.76 MB/sec execute 153 sec latency 33.693 ms 2024-09-19T06:22:50.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:49 smithi053 ceph-mon[27968]: pgmap v400: 129 pgs: 129 active+clean; 99 MiB data, 842 MiB used, 714 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.2 MiB/s wr, 106 op/s 2024-09-19T06:22:50.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:49 smithi155 ceph-mon[35292]: pgmap v400: 129 pgs: 129 active+clean; 99 MiB data, 842 MiB used, 714 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.2 MiB/s wr, 106 op/s 2024-09-19T06:22:50.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 209944 4.09 MB/sec execute 157 sec latency 70.370 ms 2024-09-19T06:22:50.695 INFO:tasks.workunit.client.0.smithi053.stdout: 1 682534 13.70 MB/sec execute 154 sec latency 291.901 ms 2024-09-19T06:22:51.623 INFO:tasks.workunit.client.1.smithi155.stdout: 1 210740 4.10 MB/sec execute 158 sec latency 64.687 ms 2024-09-19T06:22:51.695 INFO:tasks.workunit.client.0.smithi053.stdout: 1 682744 13.62 MB/sec execute 155 sec latency 258.590 ms 2024-09-19T06:22:52.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:51 smithi053 ceph-mon[27968]: pgmap v401: 129 pgs: 129 active+clean; 99 MiB data, 842 MiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.6 MiB/s wr, 80 op/s 2024-09-19T06:22:52.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:51 smithi155 ceph-mon[35292]: pgmap v401: 129 pgs: 129 active+clean; 99 MiB data, 842 MiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.6 MiB/s wr, 80 op/s 2024-09-19T06:22:52.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 211617 4.08 MB/sec execute 159 sec latency 15.063 ms 2024-09-19T06:22:52.695 INFO:tasks.workunit.client.0.smithi053.stdout: 1 686209 13.65 MB/sec execute 156 sec latency 33.523 ms 2024-09-19T06:22:53.623 INFO:tasks.workunit.client.1.smithi155.stdout: 1 212573 4.10 MB/sec execute 160 sec latency 13.829 ms 2024-09-19T06:22:53.695 INFO:tasks.workunit.client.0.smithi053.stdout: 1 689693 13.69 MB/sec execute 157 sec latency 33.626 ms 2024-09-19T06:22:54.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:54 smithi053 ceph-mon[27968]: pgmap v402: 129 pgs: 129 active+clean; 99 MiB data, 847 MiB used, 714 GiB / 715 GiB avail; 5.4 KiB/s rd, 1.9 MiB/s wr, 104 op/s 2024-09-19T06:22:54.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:54 smithi155 ceph-mon[35292]: pgmap v402: 129 pgs: 129 active+clean; 99 MiB data, 847 MiB used, 714 GiB / 715 GiB avail; 5.4 KiB/s rd, 1.9 MiB/s wr, 104 op/s 2024-09-19T06:22:54.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 213168 4.11 MB/sec execute 161 sec latency 13.888 ms 2024-09-19T06:22:54.695 INFO:tasks.workunit.client.0.smithi053.stdout: 1 693224 13.72 MB/sec execute 158 sec latency 48.512 ms 2024-09-19T06:22:55.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 213763 4.09 MB/sec execute 162 sec latency 20.264 ms 2024-09-19T06:22:55.696 INFO:tasks.workunit.client.0.smithi053.stdout: 1 696722 13.75 MB/sec execute 159 sec latency 34.347 ms 2024-09-19T06:22:56.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:56 smithi053 ceph-mon[27968]: pgmap v403: 129 pgs: 129 active+clean; 99 MiB data, 847 MiB used, 714 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.9 MiB/s wr, 85 op/s 2024-09-19T06:22:56.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:56 smithi155 ceph-mon[35292]: pgmap v403: 129 pgs: 129 active+clean; 99 MiB data, 847 MiB used, 714 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.9 MiB/s wr, 85 op/s 2024-09-19T06:22:56.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 214568 4.09 MB/sec execute 163 sec latency 52.568 ms 2024-09-19T06:22:56.695 INFO:tasks.workunit.client.0.smithi053.stdout: 1 698386 13.71 MB/sec execute 160 sec latency 200.280 ms 2024-09-19T06:22:57.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 215377 4.08 MB/sec execute 164 sec latency 19.768 ms 2024-09-19T06:22:57.695 INFO:tasks.workunit.client.0.smithi053.stdout: 1 698806 13.65 MB/sec execute 161 sec latency 300.382 ms 2024-09-19T06:22:58.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:58 smithi053 ceph-mon[27968]: pgmap v404: 129 pgs: 129 active+clean; 99 MiB data, 848 MiB used, 714 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.7 MiB/s wr, 83 op/s 2024-09-19T06:22:58.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:58 smithi155 ceph-mon[35292]: pgmap v404: 129 pgs: 129 active+clean; 99 MiB data, 848 MiB used, 714 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.7 MiB/s wr, 83 op/s 2024-09-19T06:22:58.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 216202 4.09 MB/sec execute 165 sec latency 40.137 ms 2024-09-19T06:22:58.696 INFO:tasks.workunit.client.0.smithi053.stdout: 1 700392 13.62 MB/sec execute 162 sec latency 281.234 ms 2024-09-19T06:22:59.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:22:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:22:59.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:22:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:22:59.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 216883 4.10 MB/sec execute 166 sec latency 10.842 ms 2024-09-19T06:22:59.695 INFO:tasks.workunit.client.0.smithi053.stdout: 1 703931 13.65 MB/sec execute 163 sec latency 33.181 ms 2024-09-19T06:23:00.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:00 smithi053 ceph-mon[27968]: pgmap v405: 129 pgs: 129 active+clean; 100 MiB data, 864 MiB used, 714 GiB / 715 GiB avail; 5.3 KiB/s rd, 2.1 MiB/s wr, 99 op/s 2024-09-19T06:23:00.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:00 smithi155 ceph-mon[35292]: pgmap v405: 129 pgs: 129 active+clean; 100 MiB data, 864 MiB used, 714 GiB / 715 GiB avail; 5.3 KiB/s rd, 2.1 MiB/s wr, 99 op/s 2024-09-19T06:23:00.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 217590 4.09 MB/sec execute 167 sec latency 20.431 ms 2024-09-19T06:23:00.695 INFO:tasks.workunit.client.0.smithi053.stdout: 1 707359 13.68 MB/sec execute 164 sec latency 108.159 ms 2024-09-19T06:23:01.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 218434 4.08 MB/sec execute 168 sec latency 14.627 ms 2024-09-19T06:23:01.695 INFO:tasks.workunit.client.0.smithi053.stdout: 1 709867 13.68 MB/sec execute 165 sec latency 106.246 ms 2024-09-19T06:23:02.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:02 smithi053 ceph-mon[27968]: pgmap v406: 129 pgs: 129 active+clean; 100 MiB data, 865 MiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.6 MiB/s wr, 85 op/s 2024-09-19T06:23:02.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:02 smithi155 ceph-mon[35292]: pgmap v406: 129 pgs: 129 active+clean; 100 MiB data, 865 MiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.6 MiB/s wr, 85 op/s 2024-09-19T06:23:02.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 219318 4.10 MB/sec execute 169 sec latency 15.011 ms 2024-09-19T06:23:02.695 INFO:tasks.workunit.client.0.smithi053.stdout: 1 712702 13.68 MB/sec execute 166 sec latency 53.572 ms 2024-09-19T06:23:03.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 220081 4.11 MB/sec execute 170 sec latency 14.109 ms 2024-09-19T06:23:03.695 INFO:tasks.workunit.client.0.smithi053.stdout: 1 714757 13.67 MB/sec execute 167 sec latency 107.946 ms 2024-09-19T06:23:04.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:04 smithi053 ceph-mon[27968]: pgmap v407: 129 pgs: 129 active+clean; 101 MiB data, 850 MiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 1.9 MiB/s wr, 93 op/s 2024-09-19T06:23:04.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:04 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:23:04.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:04 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:23:04.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:04 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:23:04.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:04 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:23:04.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:04 smithi155 ceph-mon[35292]: pgmap v407: 129 pgs: 129 active+clean; 101 MiB data, 850 MiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 1.9 MiB/s wr, 93 op/s 2024-09-19T06:23:04.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:04 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:23:04.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:04 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:23:04.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:04 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:23:04.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:04 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:23:04.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 220568 4.09 MB/sec execute 171 sec latency 41.964 ms 2024-09-19T06:23:04.696 INFO:tasks.workunit.client.0.smithi053.stdout: 1 715405 13.62 MB/sec execute 168 sec latency 332.847 ms 2024-09-19T06:23:05.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 221351 4.10 MB/sec execute 172 sec latency 39.207 ms 2024-09-19T06:23:05.696 INFO:tasks.workunit.client.0.smithi053.stdout: 1 718051 13.62 MB/sec execute 169 sec latency 273.928 ms 2024-09-19T06:23:06.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:06 smithi053 ceph-mon[27968]: pgmap v408: 129 pgs: 129 active+clean; 102 MiB data, 875 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.9 MiB/s wr, 93 op/s 2024-09-19T06:23:06.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:06 smithi155 ceph-mon[35292]: pgmap v408: 129 pgs: 129 active+clean; 102 MiB data, 875 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.9 MiB/s wr, 93 op/s 2024-09-19T06:23:06.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 222146 4.08 MB/sec execute 173 sec latency 14.105 ms 2024-09-19T06:23:06.696 INFO:tasks.workunit.client.0.smithi053.stdout: 1 721545 13.65 MB/sec execute 170 sec latency 32.278 ms 2024-09-19T06:23:07.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 223138 4.10 MB/sec execute 174 sec latency 16.841 ms 2024-09-19T06:23:07.696 INFO:tasks.workunit.client.0.smithi053.stdout: 1 725085 13.68 MB/sec execute 171 sec latency 41.908 ms 2024-09-19T06:23:08.284 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:08 smithi155 ceph-mon[35292]: pgmap v409: 129 pgs: 129 active+clean; 102 MiB data, 875 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 89 op/s 2024-09-19T06:23:08.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:08 smithi053 ceph-mon[27968]: pgmap v409: 129 pgs: 129 active+clean; 102 MiB data, 875 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 89 op/s 2024-09-19T06:23:08.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 223755 4.10 MB/sec execute 175 sec latency 13.377 ms 2024-09-19T06:23:08.696 INFO:tasks.workunit.client.0.smithi053.stdout: 1 727706 13.69 MB/sec execute 172 sec latency 54.005 ms 2024-09-19T06:23:09.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 224316 4.09 MB/sec execute 176 sec latency 15.806 ms 2024-09-19T06:23:09.696 INFO:tasks.workunit.client.0.smithi053.stdout: 1 730944 13.71 MB/sec execute 173 sec latency 41.667 ms 2024-09-19T06:23:10.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:10 smithi053 ceph-mon[27968]: pgmap v410: 129 pgs: 129 active+clean; 102 MiB data, 883 MiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.0 MiB/s wr, 94 op/s 2024-09-19T06:23:10.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:10 smithi155 ceph-mon[35292]: pgmap v410: 129 pgs: 129 active+clean; 102 MiB data, 883 MiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.0 MiB/s wr, 94 op/s 2024-09-19T06:23:10.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 225129 4.09 MB/sec execute 177 sec latency 15.814 ms 2024-09-19T06:23:10.696 INFO:tasks.workunit.client.0.smithi053.stdout: 1 732301 13.67 MB/sec execute 174 sec latency 258.620 ms 2024-09-19T06:23:11.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 225886 4.08 MB/sec execute 178 sec latency 42.071 ms 2024-09-19T06:23:11.696 INFO:tasks.workunit.client.0.smithi053.stdout: 1 732485 13.59 MB/sec execute 175 sec latency 316.856 ms 2024-09-19T06:23:12.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:12 smithi053 ceph-mon[27968]: pgmap v411: 129 pgs: 129 active+clean; 103 MiB data, 868 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 91 op/s 2024-09-19T06:23:12.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:12 smithi155 ceph-mon[35292]: pgmap v411: 129 pgs: 129 active+clean; 103 MiB data, 868 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 91 op/s 2024-09-19T06:23:12.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 226816 4.10 MB/sec execute 179 sec latency 18.210 ms 2024-09-19T06:23:12.696 INFO:tasks.workunit.client.0.smithi053.stdout: 1 735838 13.62 MB/sec execute 176 sec latency 108.333 ms 2024-09-19T06:23:13.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 227419 4.10 MB/sec execute 180 sec latency 12.083 ms 2024-09-19T06:23:13.696 INFO:tasks.workunit.client.0.smithi053.stdout: 1 739328 13.65 MB/sec execute 177 sec latency 25.200 ms 2024-09-19T06:23:14.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:14 smithi053 ceph-mon[27968]: pgmap v412: 129 pgs: 129 active+clean; 102 MiB data, 867 MiB used, 714 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.8 MiB/s wr, 87 op/s 2024-09-19T06:23:14.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:14 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:23:14.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:14 smithi155 ceph-mon[35292]: pgmap v412: 129 pgs: 129 active+clean; 102 MiB data, 867 MiB used, 714 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.8 MiB/s wr, 87 op/s 2024-09-19T06:23:14.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:14 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:23:14.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 228014 4.08 MB/sec execute 181 sec latency 23.553 ms 2024-09-19T06:23:14.696 INFO:tasks.workunit.client.0.smithi053.stdout: 1 742866 13.68 MB/sec execute 178 sec latency 33.431 ms 2024-09-19T06:23:15.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 228882 4.08 MB/sec execute 182 sec latency 11.985 ms 2024-09-19T06:23:15.696 INFO:tasks.workunit.client.0.smithi053.stdout: 1 746353 13.71 MB/sec execute 179 sec latency 125.082 ms 2024-09-19T06:23:16.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:16 smithi053 ceph-mon[27968]: pgmap v413: 129 pgs: 129 active+clean; 104 MiB data, 872 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 2.0 MiB/s wr, 99 op/s 2024-09-19T06:23:16.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:16 smithi155 ceph-mon[35292]: pgmap v413: 129 pgs: 129 active+clean; 104 MiB data, 872 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 2.0 MiB/s wr, 99 op/s 2024-09-19T06:23:16.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 229646 4.08 MB/sec execute 183 sec latency 13.048 ms 2024-09-19T06:23:16.696 INFO:tasks.workunit.client.0.smithi053.stdout: 1 748674 13.71 MB/sec execute 180 sec latency 91.897 ms 2024-09-19T06:23:17.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 230447 4.09 MB/sec execute 184 sec latency 13.140 ms 2024-09-19T06:23:17.696 INFO:tasks.workunit.client.0.smithi053.stdout: 1 751636 13.71 MB/sec execute 181 sec latency 34.739 ms 2024-09-19T06:23:18.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:18 smithi053 ceph-mon[27968]: pgmap v414: 129 pgs: 129 active+clean; 104 MiB data, 877 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.7 MiB/s wr, 77 op/s 2024-09-19T06:23:18.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:18 smithi155 ceph-mon[35292]: pgmap v414: 129 pgs: 129 active+clean; 104 MiB data, 877 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.7 MiB/s wr, 77 op/s 2024-09-19T06:23:18.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 231105 4.09 MB/sec execute 185 sec latency 13.636 ms 2024-09-19T06:23:18.696 INFO:tasks.workunit.client.0.smithi053.stdout: 1 754661 13.73 MB/sec execute 182 sec latency 41.112 ms 2024-09-19T06:23:19.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 231842 4.08 MB/sec execute 186 sec latency 14.839 ms 2024-09-19T06:23:19.696 INFO:tasks.workunit.client.0.smithi053.stdout: 1 757805 13.75 MB/sec execute 183 sec latency 50.213 ms 2024-09-19T06:23:20.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:20 smithi053 ceph-mon[27968]: pgmap v415: 129 pgs: 129 active+clean; 106 MiB data, 865 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 2.2 MiB/s wr, 100 op/s 2024-09-19T06:23:20.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:20 smithi155 ceph-mon[35292]: pgmap v415: 129 pgs: 129 active+clean; 106 MiB data, 865 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 2.2 MiB/s wr, 100 op/s 2024-09-19T06:23:20.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 232687 4.08 MB/sec execute 187 sec latency 14.926 ms 2024-09-19T06:23:20.696 INFO:tasks.workunit.client.0.smithi053.stdout: 1 760676 13.76 MB/sec execute 184 sec latency 83.261 ms 2024-09-19T06:23:21.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 233577 4.09 MB/sec execute 188 sec latency 12.175 ms 2024-09-19T06:23:21.696 INFO:tasks.workunit.client.0.smithi053.stdout: 1 764215 13.79 MB/sec execute 185 sec latency 26.721 ms 2024-09-19T06:23:22.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:22 smithi053 ceph-mon[27968]: pgmap v416: 129 pgs: 129 active+clean; 106 MiB data, 865 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.8 MiB/s wr, 92 op/s 2024-09-19T06:23:22.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:22 smithi155 ceph-mon[35292]: pgmap v416: 129 pgs: 129 active+clean; 106 MiB data, 865 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.8 MiB/s wr, 92 op/s 2024-09-19T06:23:22.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 234307 4.10 MB/sec execute 189 sec latency 14.504 ms 2024-09-19T06:23:22.696 INFO:tasks.workunit.client.0.smithi053.stdout: 1 767701 13.81 MB/sec execute 186 sec latency 43.002 ms 2024-09-19T06:23:23.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 234870 4.09 MB/sec execute 190 sec latency 16.742 ms 2024-09-19T06:23:23.696 INFO:tasks.workunit.client.0.smithi053.stdout: 1 771232 13.84 MB/sec execute 187 sec latency 41.827 ms 2024-09-19T06:23:24.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:24 smithi053 ceph-mon[27968]: pgmap v417: 129 pgs: 129 active+clean; 105 MiB data, 870 MiB used, 714 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.8 MiB/s wr, 84 op/s 2024-09-19T06:23:24.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:24 smithi155 ceph-mon[35292]: pgmap v417: 129 pgs: 129 active+clean; 105 MiB data, 870 MiB used, 714 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.8 MiB/s wr, 84 op/s 2024-09-19T06:23:24.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 235676 4.09 MB/sec execute 191 sec latency 11.874 ms 2024-09-19T06:23:24.696 INFO:tasks.workunit.client.0.smithi053.stdout: 1 774734 13.87 MB/sec execute 188 sec latency 33.466 ms 2024-09-19T06:23:25.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 236511 4.08 MB/sec execute 192 sec latency 21.825 ms 2024-09-19T06:23:25.697 INFO:tasks.workunit.client.0.smithi053.stdout: 1 777383 13.87 MB/sec execute 189 sec latency 33.731 ms 2024-09-19T06:23:26.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:26 smithi053 ceph-mon[27968]: pgmap v418: 129 pgs: 129 active+clean; 108 MiB data, 874 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.1 MiB/s wr, 99 op/s 2024-09-19T06:23:26.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:26 smithi155 ceph-mon[35292]: pgmap v418: 129 pgs: 129 active+clean; 108 MiB data, 874 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.1 MiB/s wr, 99 op/s 2024-09-19T06:23:26.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 237454 4.10 MB/sec execute 193 sec latency 10.542 ms 2024-09-19T06:23:26.696 INFO:tasks.workunit.client.0.smithi053.stdout: 1 780570 13.89 MB/sec execute 190 sec latency 50.080 ms 2024-09-19T06:23:27.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 238085 4.10 MB/sec execute 194 sec latency 9.475 ms 2024-09-19T06:23:27.696 INFO:tasks.workunit.client.0.smithi053.stdout: 1 783029 13.88 MB/sec execute 191 sec latency 108.474 ms 2024-09-19T06:23:28.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:28 smithi053 ceph-mon[27968]: pgmap v419: 129 pgs: 129 active+clean; 108 MiB data, 873 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 MiB/s wr, 83 op/s 2024-09-19T06:23:28.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:28 smithi155 ceph-mon[35292]: pgmap v419: 129 pgs: 129 active+clean; 108 MiB data, 873 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 MiB/s wr, 83 op/s 2024-09-19T06:23:28.624 INFO:tasks.workunit.client.1.smithi155.stdout: 1 238756 4.09 MB/sec execute 195 sec latency 16.527 ms 2024-09-19T06:23:28.696 INFO:tasks.workunit.client.0.smithi053.stdout: 1 785674 13.88 MB/sec execute 192 sec latency 36.163 ms 2024-09-19T06:23:29.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:23:29.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:29 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:23:29.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 239590 4.08 MB/sec execute 196 sec latency 14.659 ms 2024-09-19T06:23:29.697 INFO:tasks.workunit.client.0.smithi053.stdout: 1 789116 13.91 MB/sec execute 193 sec latency 33.073 ms 2024-09-19T06:23:30.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:30 smithi053 ceph-mon[27968]: pgmap v420: 129 pgs: 129 active+clean; 111 MiB data, 888 MiB used, 714 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.2 MiB/s wr, 106 op/s 2024-09-19T06:23:30.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:30 smithi155 ceph-mon[35292]: pgmap v420: 129 pgs: 129 active+clean; 111 MiB data, 888 MiB used, 714 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.2 MiB/s wr, 106 op/s 2024-09-19T06:23:30.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 240413 4.09 MB/sec execute 197 sec latency 15.416 ms 2024-09-19T06:23:30.697 INFO:tasks.workunit.client.0.smithi053.stdout: 1 792580 13.94 MB/sec execute 194 sec latency 41.784 ms 2024-09-19T06:23:31.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 241219 4.10 MB/sec execute 198 sec latency 13.877 ms 2024-09-19T06:23:31.697 INFO:tasks.workunit.client.0.smithi053.stdout: 1 796065 13.96 MB/sec execute 195 sec latency 41.811 ms 2024-09-19T06:23:32.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:32 smithi053 ceph-mon[27968]: pgmap v421: 129 pgs: 129 active+clean; 109 MiB data, 888 MiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.6 MiB/s wr, 80 op/s 2024-09-19T06:23:32.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:32 smithi155 ceph-mon[35292]: pgmap v421: 129 pgs: 129 active+clean; 109 MiB data, 888 MiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.6 MiB/s wr, 80 op/s 2024-09-19T06:23:32.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 241810 4.09 MB/sec execute 199 sec latency 10.045 ms 2024-09-19T06:23:32.697 INFO:tasks.workunit.client.0.smithi053.stdout: 1 797739 13.92 MB/sec execute 196 sec latency 305.321 ms 2024-09-19T06:23:33.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 242484 4.08 MB/sec execute 200 sec latency 48.996 ms 2024-09-19T06:23:33.697 INFO:tasks.workunit.client.0.smithi053.stdout: 1 800502 13.94 MB/sec execute 197 sec latency 91.987 ms 2024-09-19T06:23:34.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:34 smithi053 ceph-mon[27968]: pgmap v422: 129 pgs: 129 active+clean; 111 MiB data, 892 MiB used, 714 GiB / 715 GiB avail; 5.1 KiB/s rd, 1.9 MiB/s wr, 88 op/s 2024-09-19T06:23:34.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:34 smithi155 ceph-mon[35292]: pgmap v422: 129 pgs: 129 active+clean; 111 MiB data, 892 MiB used, 714 GiB / 715 GiB avail; 5.1 KiB/s rd, 1.9 MiB/s wr, 88 op/s 2024-09-19T06:23:34.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 243339 4.08 MB/sec execute 201 sec latency 17.592 ms 2024-09-19T06:23:34.697 INFO:tasks.workunit.client.0.smithi053.stdout: 1 803338 13.95 MB/sec execute 198 sec latency 40.601 ms 2024-09-19T06:23:35.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 244232 4.09 MB/sec execute 202 sec latency 12.122 ms 2024-09-19T06:23:35.697 INFO:tasks.workunit.client.0.smithi053.stdout: 1 806767 13.97 MB/sec execute 199 sec latency 33.432 ms 2024-09-19T06:23:36.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:36 smithi053 ceph-mon[27968]: pgmap v423: 129 pgs: 129 active+clean; 111 MiB data, 896 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.1 MiB/s wr, 106 op/s 2024-09-19T06:23:36.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:36 smithi155 ceph-mon[35292]: pgmap v423: 129 pgs: 129 active+clean; 111 MiB data, 896 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.1 MiB/s wr, 106 op/s 2024-09-19T06:23:36.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 245014 4.10 MB/sec execute 203 sec latency 13.099 ms 2024-09-19T06:23:36.697 INFO:tasks.workunit.client.0.smithi053.stdout: 1 810236 13.99 MB/sec execute 200 sec latency 175.090 ms 2024-09-19T06:23:37.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 245573 4.09 MB/sec execute 204 sec latency 17.864 ms 2024-09-19T06:23:37.697 INFO:tasks.workunit.client.0.smithi053.stdout: 1 813726 14.02 MB/sec execute 201 sec latency 41.742 ms 2024-09-19T06:23:38.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:38 smithi053 ceph-mon[27968]: pgmap v424: 129 pgs: 129 active+clean; 111 MiB data, 900 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 MiB/s wr, 83 op/s 2024-09-19T06:23:38.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:38 smithi155 ceph-mon[35292]: pgmap v424: 129 pgs: 129 active+clean; 111 MiB data, 900 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 MiB/s wr, 83 op/s 2024-09-19T06:23:38.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 246337 4.09 MB/sec execute 205 sec latency 33.286 ms 2024-09-19T06:23:38.780 INFO:tasks.workunit.client.0.smithi053.stdout: 1 813784 13.95 MB/sec execute 202 sec latency 300.236 ms 2024-09-19T06:23:39.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 247155 4.08 MB/sec execute 206 sec latency 26.315 ms 2024-09-19T06:23:39.697 INFO:tasks.workunit.client.0.smithi053.stdout: 1 813847 13.88 MB/sec execute 203 sec latency 266.857 ms 2024-09-19T06:23:40.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:40 smithi053 ceph-mon[27968]: pgmap v425: 129 pgs: 129 active+clean; 112 MiB data, 910 MiB used, 714 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.2 MiB/s wr, 108 op/s 2024-09-19T06:23:40.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:40 smithi155 ceph-mon[35292]: pgmap v425: 129 pgs: 129 active+clean; 112 MiB data, 910 MiB used, 714 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.2 MiB/s wr, 108 op/s 2024-09-19T06:23:40.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 248070 4.10 MB/sec execute 207 sec latency 75.492 ms 2024-09-19T06:23:40.697 INFO:tasks.workunit.client.0.smithi053.stdout: 1 813854 13.81 MB/sec execute 204 sec latency 333.559 ms 2024-09-19T06:23:41.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 248613 4.10 MB/sec execute 208 sec latency 104.320 ms 2024-09-19T06:23:41.697 INFO:tasks.workunit.client.0.smithi053.stdout: 1 813968 13.75 MB/sec execute 205 sec latency 299.847 ms 2024-09-19T06:23:42.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:42 smithi053 ceph-mon[27968]: pgmap v426: 129 pgs: 129 active+clean; 112 MiB data, 910 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.5 MiB/s wr, 83 op/s 2024-09-19T06:23:42.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:42 smithi155 ceph-mon[35292]: pgmap v426: 129 pgs: 129 active+clean; 112 MiB data, 910 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.5 MiB/s wr, 83 op/s 2024-09-19T06:23:42.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 249193 4.08 MB/sec execute 209 sec latency 38.019 ms 2024-09-19T06:23:42.697 INFO:tasks.workunit.client.0.smithi053.stdout: 1 814037 13.68 MB/sec execute 206 sec latency 300.085 ms 2024-09-19T06:23:43.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 249945 4.08 MB/sec execute 210 sec latency 48.437 ms 2024-09-19T06:23:43.697 INFO:tasks.workunit.client.0.smithi053.stdout: 1 814934 13.64 MB/sec execute 207 sec latency 307.674 ms 2024-09-19T06:23:44.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:44 smithi053 ceph-mon[27968]: pgmap v427: 129 pgs: 129 active+clean; 113 MiB data, 893 MiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 1.9 MiB/s wr, 100 op/s 2024-09-19T06:23:44.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:44 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:23:44.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:44 smithi155 ceph-mon[35292]: pgmap v427: 129 pgs: 129 active+clean; 113 MiB data, 893 MiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 1.9 MiB/s wr, 100 op/s 2024-09-19T06:23:44.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:44 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:23:44.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 250730 4.07 MB/sec execute 211 sec latency 46.041 ms 2024-09-19T06:23:44.697 INFO:tasks.workunit.client.0.smithi053.stdout: 1 815390 13.58 MB/sec execute 208 sec latency 325.238 ms 2024-09-19T06:23:45.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 251643 4.09 MB/sec execute 212 sec latency 12.211 ms 2024-09-19T06:23:45.697 INFO:tasks.workunit.client.0.smithi053.stdout: 1 817406 13.58 MB/sec execute 209 sec latency 264.446 ms 2024-09-19T06:23:46.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:46 smithi053 ceph-mon[27968]: pgmap v428: 129 pgs: 129 active+clean; 113 MiB data, 905 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.9 MiB/s wr, 102 op/s 2024-09-19T06:23:46.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:46 smithi155 ceph-mon[35292]: pgmap v428: 129 pgs: 129 active+clean; 113 MiB data, 905 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.9 MiB/s wr, 102 op/s 2024-09-19T06:23:46.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 252286 4.09 MB/sec execute 213 sec latency 11.050 ms 2024-09-19T06:23:46.697 INFO:tasks.workunit.client.0.smithi053.stdout: 1 820868 13.60 MB/sec execute 210 sec latency 129.306 ms 2024-09-19T06:23:47.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 252935 4.08 MB/sec execute 214 sec latency 17.461 ms 2024-09-19T06:23:47.697 INFO:tasks.workunit.client.0.smithi053.stdout: 1 823430 13.60 MB/sec execute 211 sec latency 108.622 ms 2024-09-19T06:23:48.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:48 smithi053 ceph-mon[27968]: pgmap v429: 129 pgs: 129 active+clean; 113 MiB data, 906 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.7 MiB/s wr, 78 op/s 2024-09-19T06:23:48.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:48 smithi155 ceph-mon[35292]: pgmap v429: 129 pgs: 129 active+clean; 113 MiB data, 906 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.7 MiB/s wr, 78 op/s 2024-09-19T06:23:48.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 253745 4.08 MB/sec execute 215 sec latency 14.852 ms 2024-09-19T06:23:48.697 INFO:tasks.workunit.client.0.smithi053.stdout: 1 826450 13.61 MB/sec execute 212 sec latency 88.673 ms 2024-09-19T06:23:49.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 254475 4.07 MB/sec execute 216 sec latency 14.400 ms 2024-09-19T06:23:49.697 INFO:tasks.workunit.client.0.smithi053.stdout: 1 829122 13.61 MB/sec execute 213 sec latency 33.546 ms 2024-09-19T06:23:50.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:50 smithi053 ceph-mon[27968]: pgmap v430: 129 pgs: 129 active+clean; 114 MiB data, 913 MiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.1 MiB/s wr, 104 op/s 2024-09-19T06:23:50.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:50 smithi155 ceph-mon[35292]: pgmap v430: 129 pgs: 129 active+clean; 114 MiB data, 913 MiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.1 MiB/s wr, 104 op/s 2024-09-19T06:23:50.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 255218 4.08 MB/sec execute 217 sec latency 48.005 ms 2024-09-19T06:23:50.697 INFO:tasks.workunit.client.0.smithi053.stdout: 1 831464 13.61 MB/sec execute 214 sec latency 292.040 ms 2024-09-19T06:23:51.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 255881 4.09 MB/sec execute 218 sec latency 15.888 ms 2024-09-19T06:23:51.697 INFO:tasks.workunit.client.0.smithi053.stdout: 1 831513 13.55 MB/sec execute 215 sec latency 283.466 ms 2024-09-19T06:23:52.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:52 smithi053 ceph-mon[27968]: pgmap v431: 129 pgs: 129 active+clean; 114 MiB data, 918 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.5 MiB/s wr, 75 op/s 2024-09-19T06:23:52.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:52 smithi155 ceph-mon[35292]: pgmap v431: 129 pgs: 129 active+clean; 114 MiB data, 918 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.5 MiB/s wr, 75 op/s 2024-09-19T06:23:52.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 256513 4.08 MB/sec execute 219 sec latency 26.506 ms 2024-09-19T06:23:52.697 INFO:tasks.workunit.client.0.smithi053.stdout: 1 835000 13.58 MB/sec execute 216 sec latency 225.168 ms 2024-09-19T06:23:53.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 257296 4.07 MB/sec execute 220 sec latency 13.282 ms 2024-09-19T06:23:53.697 INFO:tasks.workunit.client.0.smithi053.stdout: 1 837652 13.58 MB/sec execute 217 sec latency 66.729 ms 2024-09-19T06:23:54.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:54 smithi053 ceph-mon[27968]: pgmap v432: 129 pgs: 129 active+clean; 115 MiB data, 919 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 1.9 MiB/s wr, 100 op/s 2024-09-19T06:23:54.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:54 smithi155 ceph-mon[35292]: pgmap v432: 129 pgs: 129 active+clean; 115 MiB data, 919 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 1.9 MiB/s wr, 100 op/s 2024-09-19T06:23:54.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 258012 4.07 MB/sec execute 221 sec latency 17.611 ms 2024-09-19T06:23:54.698 INFO:tasks.workunit.client.0.smithi053.stdout: 1 840673 13.59 MB/sec execute 218 sec latency 40.890 ms 2024-09-19T06:23:55.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 258753 4.08 MB/sec execute 222 sec latency 19.057 ms 2024-09-19T06:23:55.698 INFO:tasks.workunit.client.0.smithi053.stdout: 1 843819 13.60 MB/sec execute 219 sec latency 33.480 ms 2024-09-19T06:23:56.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:56 smithi053 ceph-mon[27968]: pgmap v433: 129 pgs: 129 active+clean; 116 MiB data, 944 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.9 MiB/s wr, 88 op/s 2024-09-19T06:23:56.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:56 smithi155 ceph-mon[35292]: pgmap v433: 129 pgs: 129 active+clean; 116 MiB data, 944 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.9 MiB/s wr, 88 op/s 2024-09-19T06:23:56.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 259365 4.08 MB/sec execute 223 sec latency 19.253 ms 2024-09-19T06:23:56.698 INFO:tasks.workunit.client.0.smithi053.stdout: 1 846508 13.61 MB/sec execute 220 sec latency 58.356 ms 2024-09-19T06:23:57.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 259938 4.07 MB/sec execute 224 sec latency 20.458 ms 2024-09-19T06:23:57.698 INFO:tasks.workunit.client.0.smithi053.stdout: 1 849490 13.62 MB/sec execute 221 sec latency 41.795 ms 2024-09-19T06:23:58.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:58 smithi053 ceph-mon[27968]: pgmap v434: 129 pgs: 129 active+clean; 115 MiB data, 948 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.6 MiB/s wr, 80 op/s 2024-09-19T06:23:58.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:58 smithi155 ceph-mon[35292]: pgmap v434: 129 pgs: 129 active+clean; 115 MiB data, 948 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.6 MiB/s wr, 80 op/s 2024-09-19T06:23:58.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 260714 4.07 MB/sec execute 225 sec latency 18.614 ms 2024-09-19T06:23:58.698 INFO:tasks.workunit.client.0.smithi053.stdout: 1 852920 13.64 MB/sec execute 222 sec latency 41.442 ms 2024-09-19T06:23:59.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:23:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:23:59.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:23:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:23:59.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 261404 4.06 MB/sec execute 226 sec latency 18.758 ms 2024-09-19T06:23:59.698 INFO:tasks.workunit.client.0.smithi053.stdout: 1 856459 13.67 MB/sec execute 223 sec latency 33.351 ms 2024-09-19T06:24:00.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:00 smithi053 ceph-mon[27968]: pgmap v435: 129 pgs: 129 active+clean; 117 MiB data, 935 MiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.0 MiB/s wr, 103 op/s 2024-09-19T06:24:00.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:00 smithi155 ceph-mon[35292]: pgmap v435: 129 pgs: 129 active+clean; 117 MiB data, 935 MiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.0 MiB/s wr, 103 op/s 2024-09-19T06:24:00.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 262276 4.07 MB/sec execute 227 sec latency 17.179 ms 2024-09-19T06:24:00.698 INFO:tasks.workunit.client.0.smithi053.stdout: 1 859944 13.69 MB/sec execute 224 sec latency 33.653 ms 2024-09-19T06:24:01.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 262869 4.07 MB/sec execute 228 sec latency 16.721 ms 2024-09-19T06:24:01.698 INFO:tasks.workunit.client.0.smithi053.stdout: 1 863477 13.71 MB/sec execute 225 sec latency 41.768 ms 2024-09-19T06:24:02.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:02 smithi053 ceph-mon[27968]: pgmap v436: 129 pgs: 129 active+clean; 117 MiB data, 935 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.5 MiB/s wr, 75 op/s 2024-09-19T06:24:02.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:02 smithi155 ceph-mon[35292]: pgmap v436: 129 pgs: 129 active+clean; 117 MiB data, 935 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.5 MiB/s wr, 75 op/s 2024-09-19T06:24:02.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 263459 4.06 MB/sec execute 229 sec latency 17.963 ms 2024-09-19T06:24:02.698 INFO:tasks.workunit.client.0.smithi053.stdout: 1 866978 13.74 MB/sec execute 226 sec latency 33.430 ms 2024-09-19T06:24:03.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 264225 4.06 MB/sec execute 230 sec latency 15.352 ms 2024-09-19T06:24:03.698 INFO:tasks.workunit.client.0.smithi053.stdout: 1 870514 13.76 MB/sec execute 227 sec latency 35.218 ms 2024-09-19T06:24:04.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:04 smithi053 ceph-mon[27968]: pgmap v437: 129 pgs: 129 active+clean; 118 MiB data, 927 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 1.8 MiB/s wr, 100 op/s 2024-09-19T06:24:04.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:04 smithi155 ceph-mon[35292]: pgmap v437: 129 pgs: 129 active+clean; 118 MiB data, 927 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 1.8 MiB/s wr, 100 op/s 2024-09-19T06:24:04.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 264948 4.05 MB/sec execute 231 sec latency 19.478 ms 2024-09-19T06:24:04.698 INFO:tasks.workunit.client.0.smithi053.stdout: 1 873139 13.76 MB/sec execute 228 sec latency 44.224 ms 2024-09-19T06:24:05.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:05 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:24:05.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:05 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:24:05.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:05 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:24:05.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:05 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:24:05.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:05 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:24:05.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:05 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:24:05.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:05 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:24:05.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:05 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:24:05.625 INFO:tasks.workunit.client.1.smithi155.stdout: 1 265821 4.07 MB/sec execute 232 sec latency 18.195 ms 2024-09-19T06:24:05.698 INFO:tasks.workunit.client.0.smithi053.stdout: 1 876304 13.78 MB/sec execute 229 sec latency 50.181 ms 2024-09-19T06:24:06.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 266398 4.07 MB/sec execute 233 sec latency 19.014 ms 2024-09-19T06:24:06.698 INFO:tasks.workunit.client.0.smithi053.stdout: 1 879304 13.78 MB/sec execute 230 sec latency 48.128 ms 2024-09-19T06:24:06.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:06 smithi053 ceph-mon[27968]: pgmap v438: 129 pgs: 129 active+clean; 119 MiB data, 927 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.8 MiB/s wr, 81 op/s 2024-09-19T06:24:06.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:06 smithi155 ceph-mon[35292]: pgmap v438: 129 pgs: 129 active+clean; 119 MiB data, 927 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.8 MiB/s wr, 81 op/s 2024-09-19T06:24:07.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 267001 4.06 MB/sec execute 234 sec latency 16.237 ms 2024-09-19T06:24:07.698 INFO:tasks.workunit.client.0.smithi053.stdout: 1 881468 13.78 MB/sec execute 231 sec latency 108.410 ms 2024-09-19T06:24:08.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 267728 4.06 MB/sec execute 235 sec latency 20.449 ms 2024-09-19T06:24:08.698 INFO:tasks.workunit.client.0.smithi053.stdout: 1 884818 13.80 MB/sec execute 232 sec latency 133.367 ms 2024-09-19T06:24:08.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:08 smithi053 ceph-mon[27968]: pgmap v439: 129 pgs: 129 active+clean; 119 MiB data, 933 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.6 MiB/s wr, 79 op/s 2024-09-19T06:24:08.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:08 smithi155 ceph-mon[35292]: pgmap v439: 129 pgs: 129 active+clean; 119 MiB data, 933 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.6 MiB/s wr, 79 op/s 2024-09-19T06:24:09.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 268472 4.05 MB/sec execute 236 sec latency 22.440 ms 2024-09-19T06:24:09.698 INFO:tasks.workunit.client.0.smithi053.stdout: 1 888306 13.82 MB/sec execute 233 sec latency 52.848 ms 2024-09-19T06:24:10.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 269363 4.06 MB/sec execute 237 sec latency 18.675 ms 2024-09-19T06:24:10.698 INFO:tasks.workunit.client.0.smithi053.stdout: 1 891798 13.84 MB/sec execute 234 sec latency 27.206 ms 2024-09-19T06:24:10.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:10 smithi053 ceph-mon[27968]: pgmap v440: 129 pgs: 129 active+clean; 119 MiB data, 934 MiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.0 MiB/s wr, 101 op/s 2024-09-19T06:24:10.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:10 smithi155 ceph-mon[35292]: pgmap v440: 129 pgs: 129 active+clean; 119 MiB data, 934 MiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.0 MiB/s wr, 101 op/s 2024-09-19T06:24:11.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 269913 4.06 MB/sec execute 238 sec latency 28.382 ms 2024-09-19T06:24:11.698 INFO:tasks.workunit.client.0.smithi053.stdout: 1 895342 13.86 MB/sec execute 235 sec latency 26.876 ms 2024-09-19T06:24:12.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 270467 4.05 MB/sec execute 239 sec latency 17.727 ms 2024-09-19T06:24:12.698 INFO:tasks.workunit.client.0.smithi053.stdout: 1 897004 13.83 MB/sec execute 236 sec latency 308.591 ms 2024-09-19T06:24:12.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:12 smithi053 ceph-mon[27968]: pgmap v441: 129 pgs: 129 active+clean; 119 MiB data, 934 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.5 MiB/s wr, 75 op/s 2024-09-19T06:24:12.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:12 smithi155 ceph-mon[35292]: pgmap v441: 129 pgs: 129 active+clean; 119 MiB data, 934 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.5 MiB/s wr, 75 op/s 2024-09-19T06:24:13.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 271213 4.05 MB/sec execute 240 sec latency 15.886 ms 2024-09-19T06:24:13.698 INFO:tasks.workunit.client.0.smithi053.stdout: 1 899192 13.83 MB/sec execute 237 sec latency 64.768 ms 2024-09-19T06:24:14.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 272010 4.04 MB/sec execute 241 sec latency 16.924 ms 2024-09-19T06:24:14.698 INFO:tasks.workunit.client.0.smithi053.stdout: 1 902672 13.85 MB/sec execute 238 sec latency 82.105 ms 2024-09-19T06:24:14.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:14 smithi053 ceph-mon[27968]: pgmap v442: 129 pgs: 129 active+clean; 119 MiB data, 953 MiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 1.8 MiB/s wr, 94 op/s 2024-09-19T06:24:14.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:14 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:24:14.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:14 smithi155 ceph-mon[35292]: pgmap v442: 129 pgs: 129 active+clean; 119 MiB data, 953 MiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 1.8 MiB/s wr, 94 op/s 2024-09-19T06:24:14.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:14 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:24:15.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 272917 4.06 MB/sec execute 242 sec latency 12.371 ms 2024-09-19T06:24:15.698 INFO:tasks.workunit.client.0.smithi053.stdout: 1 906093 13.87 MB/sec execute 239 sec latency 66.279 ms 2024-09-19T06:24:16.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 273558 4.06 MB/sec execute 243 sec latency 12.356 ms 2024-09-19T06:24:16.698 INFO:tasks.workunit.client.0.smithi053.stdout: 1 909583 13.89 MB/sec execute 240 sec latency 33.178 ms 2024-09-19T06:24:16.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:16 smithi053 ceph-mon[27968]: pgmap v443: 129 pgs: 129 active+clean; 120 MiB data, 964 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 MiB/s wr, 77 op/s 2024-09-19T06:24:16.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:16 smithi155 ceph-mon[35292]: pgmap v443: 129 pgs: 129 active+clean; 120 MiB data, 964 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 MiB/s wr, 77 op/s 2024-09-19T06:24:17.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 274260 4.05 MB/sec execute 244 sec latency 15.747 ms 2024-09-19T06:24:17.698 INFO:tasks.workunit.client.0.smithi053.stdout: 1 913067 13.91 MB/sec execute 241 sec latency 99.045 ms 2024-09-19T06:24:18.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 275027 4.05 MB/sec execute 245 sec latency 58.040 ms 2024-09-19T06:24:18.699 INFO:tasks.workunit.client.0.smithi053.stdout: 1 913121 13.86 MB/sec execute 242 sec latency 324.892 ms 2024-09-19T06:24:18.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:18 smithi053 ceph-mon[27968]: pgmap v444: 129 pgs: 129 active+clean; 121 MiB data, 983 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 91 op/s 2024-09-19T06:24:18.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:18 smithi155 ceph-mon[35292]: pgmap v444: 129 pgs: 129 active+clean; 121 MiB data, 983 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 91 op/s 2024-09-19T06:24:19.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 275766 4.04 MB/sec execute 246 sec latency 39.492 ms 2024-09-19T06:24:19.698 INFO:tasks.workunit.client.0.smithi053.stdout: 1 913182 13.80 MB/sec execute 243 sec latency 325.130 ms 2024-09-19T06:24:20.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 276541 4.05 MB/sec execute 247 sec latency 26.495 ms 2024-09-19T06:24:20.699 INFO:tasks.workunit.client.0.smithi053.stdout: 1 913191 13.74 MB/sec execute 244 sec latency 275.278 ms 2024-09-19T06:24:20.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:20 smithi053 ceph-mon[27968]: pgmap v445: 129 pgs: 129 active+clean; 123 MiB data, 992 MiB used, 714 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.0 MiB/s wr, 97 op/s 2024-09-19T06:24:20.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:20 smithi155 ceph-mon[35292]: pgmap v445: 129 pgs: 129 active+clean; 123 MiB data, 992 MiB used, 714 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.0 MiB/s wr, 97 op/s 2024-09-19T06:24:21.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 277209 4.05 MB/sec execute 248 sec latency 36.412 ms 2024-09-19T06:24:21.699 INFO:tasks.workunit.client.0.smithi053.stdout: 1 913305 13.69 MB/sec execute 245 sec latency 308.721 ms 2024-09-19T06:24:22.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 277870 4.05 MB/sec execute 249 sec latency 95.737 ms 2024-09-19T06:24:22.699 INFO:tasks.workunit.client.0.smithi053.stdout: 1 913374 13.63 MB/sec execute 246 sec latency 291.429 ms 2024-09-19T06:24:22.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:22 smithi053 ceph-mon[27968]: pgmap v446: 129 pgs: 129 active+clean; 122 MiB data, 990 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.5 MiB/s wr, 72 op/s 2024-09-19T06:24:22.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:22 smithi155 ceph-mon[35292]: pgmap v446: 129 pgs: 129 active+clean; 122 MiB data, 990 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.5 MiB/s wr, 72 op/s 2024-09-19T06:24:23.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 278611 4.04 MB/sec execute 250 sec latency 73.000 ms 2024-09-19T06:24:23.699 INFO:tasks.workunit.client.0.smithi053.stdout: 1 914272 13.60 MB/sec execute 247 sec latency 316.904 ms 2024-09-19T06:24:24.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 279451 4.05 MB/sec execute 251 sec latency 34.874 ms 2024-09-19T06:24:24.699 INFO:tasks.workunit.client.0.smithi053.stdout: 1 914800 13.55 MB/sec execute 248 sec latency 289.631 ms 2024-09-19T06:24:24.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:24 smithi053 ceph-mon[27968]: pgmap v447: 129 pgs: 129 active+clean; 124 MiB data, 984 MiB used, 714 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.9 MiB/s wr, 98 op/s 2024-09-19T06:24:24.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:24 smithi155 ceph-mon[35292]: pgmap v447: 129 pgs: 129 active+clean; 124 MiB data, 984 MiB used, 714 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.9 MiB/s wr, 98 op/s 2024-09-19T06:24:25.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 280251 4.06 MB/sec execute 252 sec latency 13.880 ms 2024-09-19T06:24:25.699 INFO:tasks.workunit.client.0.smithi053.stdout: 1 917671 13.63 MB/sec execute 249 sec latency 27.245 ms 2024-09-19T06:24:26.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 280843 4.05 MB/sec execute 253 sec latency 14.876 ms 2024-09-19T06:24:26.699 INFO:tasks.workunit.client.0.smithi053.stdout: 1 921048 13.65 MB/sec execute 250 sec latency 111.223 ms 2024-09-19T06:24:26.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:26 smithi053 ceph-mon[27968]: pgmap v448: 129 pgs: 129 active+clean; 124 MiB data, 986 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.9 MiB/s wr, 85 op/s 2024-09-19T06:24:26.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:26 smithi155 ceph-mon[35292]: pgmap v448: 129 pgs: 129 active+clean; 124 MiB data, 986 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.9 MiB/s wr, 85 op/s 2024-09-19T06:24:27.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 281654 4.05 MB/sec execute 254 sec latency 18.657 ms 2024-09-19T06:24:27.699 INFO:tasks.workunit.client.0.smithi053.stdout: 1 924524 13.67 MB/sec execute 251 sec latency 42.963 ms 2024-09-19T06:24:28.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 282546 4.04 MB/sec execute 255 sec latency 10.509 ms 2024-09-19T06:24:28.699 INFO:tasks.workunit.client.0.smithi053.stdout: 1 928020 13.69 MB/sec execute 252 sec latency 83.304 ms 2024-09-19T06:24:28.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:28 smithi053 ceph-mon[27968]: pgmap v449: 129 pgs: 129 active+clean; 125 MiB data, 987 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.8 MiB/s wr, 95 op/s 2024-09-19T06:24:28.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:28 smithi155 ceph-mon[35292]: pgmap v449: 129 pgs: 129 active+clean; 125 MiB data, 987 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.8 MiB/s wr, 95 op/s 2024-09-19T06:24:29.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 283524 4.06 MB/sec execute 256 sec latency 11.367 ms 2024-09-19T06:24:29.699 INFO:tasks.workunit.client.0.smithi053.stdout: 1 930666 13.70 MB/sec execute 253 sec latency 75.161 ms 2024-09-19T06:24:29.857 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:24:29.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:29 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:24:30.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 284095 4.06 MB/sec execute 257 sec latency 12.588 ms 2024-09-19T06:24:30.699 INFO:tasks.workunit.client.0.smithi053.stdout: 1 931684 13.66 MB/sec execute 254 sec latency 291.776 ms 2024-09-19T06:24:30.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:30 smithi053 ceph-mon[27968]: pgmap v450: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.1 MiB/s wr, 93 op/s 2024-09-19T06:24:30.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:30 smithi155 ceph-mon[35292]: pgmap v450: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.1 MiB/s wr, 93 op/s 2024-09-19T06:24:31.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 284658 4.05 MB/sec execute 258 sec latency 51.743 ms 2024-09-19T06:24:31.699 INFO:tasks.workunit.client.0.smithi053.stdout: 1 931691 13.61 MB/sec execute 255 sec latency 291.845 ms 2024-09-19T06:24:32.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 285470 4.05 MB/sec execute 259 sec latency 14.977 ms 2024-09-19T06:24:32.699 INFO:tasks.workunit.client.0.smithi053.stdout: 1 935176 13.63 MB/sec execute 256 sec latency 197.154 ms 2024-09-19T06:24:32.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:32 smithi053 ceph-mon[27968]: pgmap v451: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 82 op/s 2024-09-19T06:24:32.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:32 smithi155 ceph-mon[35292]: pgmap v451: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 82 op/s 2024-09-19T06:24:33.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 286279 4.04 MB/sec execute 260 sec latency 18.014 ms 2024-09-19T06:24:33.699 INFO:tasks.workunit.client.0.smithi053.stdout: 1 938745 13.65 MB/sec execute 257 sec latency 47.920 ms 2024-09-19T06:24:34.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 287117 4.05 MB/sec execute 261 sec latency 13.901 ms 2024-09-19T06:24:34.699 INFO:tasks.workunit.client.0.smithi053.stdout: 1 942244 13.67 MB/sec execute 258 sec latency 41.678 ms 2024-09-19T06:24:34.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:34 smithi053 ceph-mon[27968]: pgmap v452: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.8 MiB/s wr, 90 op/s 2024-09-19T06:24:34.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:34 smithi155 ceph-mon[35292]: pgmap v452: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.8 MiB/s wr, 90 op/s 2024-09-19T06:24:35.626 INFO:tasks.workunit.client.1.smithi155.stdout: 1 287801 4.06 MB/sec execute 262 sec latency 10.110 ms 2024-09-19T06:24:35.699 INFO:tasks.workunit.client.0.smithi053.stdout: 1 944863 13.67 MB/sec execute 259 sec latency 33.442 ms 2024-09-19T06:24:36.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 288507 4.05 MB/sec execute 263 sec latency 16.053 ms 2024-09-19T06:24:36.699 INFO:tasks.workunit.client.0.smithi053.stdout: 1 947871 13.68 MB/sec execute 260 sec latency 33.690 ms 2024-09-19T06:24:36.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:36 smithi053 ceph-mon[27968]: pgmap v453: 129 pgs: 129 active+clean; 127 MiB data, 1023 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.9 MiB/s wr, 90 op/s 2024-09-19T06:24:36.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:36 smithi155 ceph-mon[35292]: pgmap v453: 129 pgs: 129 active+clean; 127 MiB data, 1023 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.9 MiB/s wr, 90 op/s 2024-09-19T06:24:37.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 289250 4.05 MB/sec execute 264 sec latency 81.796 ms 2024-09-19T06:24:37.699 INFO:tasks.workunit.client.0.smithi053.stdout: 1 948300 13.64 MB/sec execute 261 sec latency 300.062 ms 2024-09-19T06:24:38.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 290110 4.05 MB/sec execute 265 sec latency 18.793 ms 2024-09-19T06:24:38.699 INFO:tasks.workunit.client.0.smithi053.stdout: 1 949539 13.61 MB/sec execute 262 sec latency 265.776 ms 2024-09-19T06:24:38.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:38 smithi053 ceph-mon[27968]: pgmap v454: 129 pgs: 129 active+clean; 127 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 MiB/s wr, 86 op/s 2024-09-19T06:24:38.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:38 smithi155 ceph-mon[35292]: pgmap v454: 129 pgs: 129 active+clean; 127 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 MiB/s wr, 86 op/s 2024-09-19T06:24:39.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 290939 4.06 MB/sec execute 266 sec latency 12.277 ms 2024-09-19T06:24:39.699 INFO:tasks.workunit.client.0.smithi053.stdout: 1 952930 13.63 MB/sec execute 263 sec latency 124.635 ms 2024-09-19T06:24:40.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 291489 4.05 MB/sec execute 267 sec latency 12.394 ms 2024-09-19T06:24:40.699 INFO:tasks.workunit.client.0.smithi053.stdout: 1 956420 13.65 MB/sec execute 264 sec latency 93.089 ms 2024-09-19T06:24:40.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:40 smithi053 ceph-mon[27968]: pgmap v455: 129 pgs: 129 active+clean; 128 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.0 MiB/s wr, 97 op/s 2024-09-19T06:24:40.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:40 smithi155 ceph-mon[35292]: pgmap v455: 129 pgs: 129 active+clean; 128 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.0 MiB/s wr, 97 op/s 2024-09-19T06:24:41.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 292323 4.06 MB/sec execute 268 sec latency 16.173 ms 2024-09-19T06:24:41.700 INFO:tasks.workunit.client.0.smithi053.stdout: 1 959038 13.65 MB/sec execute 265 sec latency 37.178 ms 2024-09-19T06:24:42.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 293197 4.05 MB/sec execute 269 sec latency 13.181 ms 2024-09-19T06:24:42.699 INFO:tasks.workunit.client.0.smithi053.stdout: 1 962265 13.67 MB/sec execute 266 sec latency 32.828 ms 2024-09-19T06:24:42.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:42 smithi053 ceph-mon[27968]: pgmap v456: 129 pgs: 129 active+clean; 128 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.6 MiB/s wr, 79 op/s 2024-09-19T06:24:42.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:42 smithi155 ceph-mon[35292]: pgmap v456: 129 pgs: 129 active+clean; 128 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.6 MiB/s wr, 79 op/s 2024-09-19T06:24:43.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 294162 4.06 MB/sec execute 270 sec latency 12.442 ms 2024-09-19T06:24:43.699 INFO:tasks.workunit.client.0.smithi053.stdout: 1 963719 13.64 MB/sec execute 267 sec latency 208.548 ms 2024-09-19T06:24:44.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 294648 4.06 MB/sec execute 271 sec latency 75.849 ms 2024-09-19T06:24:44.700 INFO:tasks.workunit.client.0.smithi053.stdout: 1 963825 13.59 MB/sec execute 268 sec latency 291.903 ms 2024-09-19T06:24:44.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:44 smithi053 ceph-mon[27968]: pgmap v457: 129 pgs: 129 active+clean; 127 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 5.3 KiB/s rd, 1.8 MiB/s wr, 87 op/s 2024-09-19T06:24:44.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:44 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:24:44.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:44 smithi155 ceph-mon[35292]: pgmap v457: 129 pgs: 129 active+clean; 127 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 5.3 KiB/s rd, 1.8 MiB/s wr, 87 op/s 2024-09-19T06:24:44.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:44 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:24:45.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 295168 4.05 MB/sec execute 272 sec latency 24.990 ms 2024-09-19T06:24:45.699 INFO:tasks.workunit.client.0.smithi053.stdout: 1 967111 13.61 MB/sec execute 269 sec latency 257.970 ms 2024-09-19T06:24:46.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 295961 4.05 MB/sec execute 273 sec latency 11.928 ms 2024-09-19T06:24:46.700 INFO:tasks.workunit.client.0.smithi053.stdout: 1 970608 13.63 MB/sec execute 270 sec latency 49.621 ms 2024-09-19T06:24:46.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:46 smithi053 ceph-mon[27968]: pgmap v458: 129 pgs: 129 active+clean; 129 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.7 KiB/s rd, 2.1 MiB/s wr, 105 op/s 2024-09-19T06:24:46.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:46 smithi155 ceph-mon[35292]: pgmap v458: 129 pgs: 129 active+clean; 129 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.7 KiB/s rd, 2.1 MiB/s wr, 105 op/s 2024-09-19T06:24:47.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 296818 4.04 MB/sec execute 274 sec latency 16.922 ms 2024-09-19T06:24:47.700 INFO:tasks.workunit.client.0.smithi053.stdout: 1 973107 13.63 MB/sec execute 271 sec latency 140.189 ms 2024-09-19T06:24:48.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 297739 4.06 MB/sec execute 275 sec latency 12.993 ms 2024-09-19T06:24:48.700 INFO:tasks.workunit.client.0.smithi053.stdout: 1 976233 13.64 MB/sec execute 272 sec latency 33.552 ms 2024-09-19T06:24:48.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:48 smithi053 ceph-mon[27968]: pgmap v459: 129 pgs: 129 active+clean; 129 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.7 MiB/s wr, 81 op/s 2024-09-19T06:24:48.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:48 smithi155 ceph-mon[35292]: pgmap v459: 129 pgs: 129 active+clean; 129 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.7 MiB/s wr, 81 op/s 2024-09-19T06:24:49.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 298359 4.06 MB/sec execute 276 sec latency 11.740 ms 2024-09-19T06:24:49.700 INFO:tasks.workunit.client.0.smithi053.stdout: 1 978913 13.64 MB/sec execute 273 sec latency 66.431 ms 2024-09-19T06:24:50.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 298925 4.05 MB/sec execute 277 sec latency 97.879 ms 2024-09-19T06:24:50.700 INFO:tasks.workunit.client.0.smithi053.stdout: 1 981236 13.64 MB/sec execute 274 sec latency 293.315 ms 2024-09-19T06:24:50.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:50 smithi053 ceph-mon[27968]: pgmap v460: 129 pgs: 129 active+clean; 130 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 5.3 KiB/s rd, 2.1 MiB/s wr, 106 op/s 2024-09-19T06:24:50.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:50 smithi155 ceph-mon[35292]: pgmap v460: 129 pgs: 129 active+clean; 130 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 5.3 KiB/s rd, 2.1 MiB/s wr, 106 op/s 2024-09-19T06:24:51.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 299743 4.05 MB/sec execute 278 sec latency 35.032 ms 2024-09-19T06:24:51.700 INFO:tasks.workunit.client.0.smithi053.stdout: 1 981363 13.59 MB/sec execute 275 sec latency 325.257 ms 2024-09-19T06:24:52.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 300550 4.04 MB/sec execute 279 sec latency 14.066 ms 2024-09-19T06:24:52.700 INFO:tasks.workunit.client.0.smithi053.stdout: 1 984911 13.61 MB/sec execute 276 sec latency 33.380 ms 2024-09-19T06:24:52.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:52 smithi155 ceph-mon[35292]: pgmap v461: 129 pgs: 129 active+clean; 130 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.6 MiB/s wr, 78 op/s 2024-09-19T06:24:53.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:52 smithi053 ceph-mon[27968]: pgmap v461: 129 pgs: 129 active+clean; 130 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.6 MiB/s wr, 78 op/s 2024-09-19T06:24:53.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 301366 4.05 MB/sec execute 280 sec latency 11.813 ms 2024-09-19T06:24:53.700 INFO:tasks.workunit.client.0.smithi053.stdout: 1 988390 13.63 MB/sec execute 277 sec latency 33.415 ms 2024-09-19T06:24:54.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 302067 4.05 MB/sec execute 281 sec latency 11.890 ms 2024-09-19T06:24:54.700 INFO:tasks.workunit.client.0.smithi053.stdout: 1 991920 13.65 MB/sec execute 278 sec latency 57.267 ms 2024-09-19T06:24:54.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:54 smithi155 ceph-mon[35292]: pgmap v462: 129 pgs: 129 active+clean; 131 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 5.1 KiB/s rd, 1.9 MiB/s wr, 87 op/s 2024-09-19T06:24:55.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:54 smithi053 ceph-mon[27968]: pgmap v462: 129 pgs: 129 active+clean; 131 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 5.1 KiB/s rd, 1.9 MiB/s wr, 87 op/s 2024-09-19T06:24:55.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 302905 4.06 MB/sec execute 282 sec latency 15.364 ms 2024-09-19T06:24:55.700 INFO:tasks.workunit.client.0.smithi053.stdout: 1 994526 13.65 MB/sec execute 279 sec latency 158.412 ms 2024-09-19T06:24:56.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 303722 4.05 MB/sec execute 283 sec latency 14.059 ms 2024-09-19T06:24:56.700 INFO:tasks.workunit.client.0.smithi053.stdout: 1 997192 13.65 MB/sec execute 280 sec latency 68.224 ms 2024-09-19T06:24:56.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:56 smithi155 ceph-mon[35292]: pgmap v463: 129 pgs: 129 active+clean; 133 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.1 MiB/s wr, 102 op/s 2024-09-19T06:24:57.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:56 smithi053 ceph-mon[27968]: pgmap v463: 129 pgs: 129 active+clean; 133 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.1 MiB/s wr, 102 op/s 2024-09-19T06:24:57.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 304521 4.06 MB/sec execute 284 sec latency 82.783 ms 2024-09-19T06:24:57.700 INFO:tasks.workunit.client.0.smithi053.stdout: 1 997753 13.62 MB/sec execute 281 sec latency 323.645 ms 2024-09-19T06:24:58.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 305216 4.06 MB/sec execute 285 sec latency 78.625 ms 2024-09-19T06:24:58.700 INFO:tasks.workunit.client.0.smithi053.stdout: 1 999133 13.60 MB/sec execute 282 sec latency 266.564 ms 2024-09-19T06:24:58.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:58 smithi155 ceph-mon[35292]: pgmap v464: 129 pgs: 129 active+clean; 133 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 MiB/s wr, 84 op/s 2024-09-19T06:24:59.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:58 smithi053 ceph-mon[27968]: pgmap v464: 129 pgs: 129 active+clean; 133 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 MiB/s wr, 84 op/s 2024-09-19T06:24:59.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 305788 4.05 MB/sec execute 286 sec latency 18.507 ms 2024-09-19T06:24:59.700 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1002604 13.61 MB/sec execute 283 sec latency 108.116 ms 2024-09-19T06:24:59.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:24:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:25:00.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:24:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:25:00.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 306625 4.05 MB/sec execute 287 sec latency 14.647 ms 2024-09-19T06:25:00.700 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1005192 13.62 MB/sec execute 284 sec latency 126.580 ms 2024-09-19T06:25:00.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:00 smithi155 ceph-mon[35292]: pgmap v465: 129 pgs: 129 active+clean; 133 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.1 MiB/s wr, 102 op/s 2024-09-19T06:25:01.094 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:00 smithi053 ceph-mon[27968]: pgmap v465: 129 pgs: 129 active+clean; 133 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.1 MiB/s wr, 102 op/s 2024-09-19T06:25:01.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 307477 4.05 MB/sec execute 288 sec latency 13.010 ms 2024-09-19T06:25:01.700 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1008411 13.63 MB/sec execute 285 sec latency 58.073 ms 2024-09-19T06:25:02.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 308380 4.06 MB/sec execute 289 sec latency 13.907 ms 2024-09-19T06:25:02.700 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1011336 13.63 MB/sec execute 286 sec latency 78.744 ms 2024-09-19T06:25:02.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:02 smithi155 ceph-mon[35292]: pgmap v466: 129 pgs: 129 active+clean; 132 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.5 MiB/s wr, 75 op/s 2024-09-19T06:25:03.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:02 smithi053 ceph-mon[27968]: pgmap v466: 129 pgs: 129 active+clean; 132 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.5 MiB/s wr, 75 op/s 2024-09-19T06:25:03.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 309038 4.06 MB/sec execute 290 sec latency 12.818 ms 2024-09-19T06:25:03.700 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1014394 13.64 MB/sec execute 287 sec latency 33.478 ms 2024-09-19T06:25:04.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 309735 4.05 MB/sec execute 291 sec latency 17.205 ms 2024-09-19T06:25:04.700 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1016976 13.65 MB/sec execute 288 sec latency 108.425 ms 2024-09-19T06:25:04.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:04 smithi155 ceph-mon[35292]: pgmap v467: 129 pgs: 129 active+clean; 133 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.0 MiB/s wr, 101 op/s 2024-09-19T06:25:05.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:04 smithi053 ceph-mon[27968]: pgmap v467: 129 pgs: 129 active+clean; 133 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.0 MiB/s wr, 101 op/s 2024-09-19T06:25:05.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 310549 4.05 MB/sec execute 292 sec latency 16.534 ms 2024-09-19T06:25:05.701 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1020407 13.66 MB/sec execute 289 sec latency 50.140 ms 2024-09-19T06:25:05.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:05 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:25:05.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:05 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:25:05.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:05 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:25:05.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:05 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:25:05.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:05 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:25:05.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:05 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:25:05.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:05 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:25:05.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:05 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:25:05.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:05 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:25:06.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:05 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:25:06.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:05 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:25:06.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:05 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:25:06.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:05 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:25:06.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:05 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:25:06.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:05 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:25:06.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:05 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:25:06.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:05 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:25:06.106 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:05 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:25:06.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 311483 4.06 MB/sec execute 293 sec latency 13.391 ms 2024-09-19T06:25:06.700 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1023944 13.68 MB/sec execute 290 sec latency 32.771 ms 2024-09-19T06:25:06.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:06 smithi155 ceph-mon[35292]: pgmap v468: 129 pgs: 129 active+clean; 133 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 2.0 MiB/s wr, 100 op/s 2024-09-19T06:25:06.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:06 smithi155 ceph-mon[35292]: Adjusting osd_memory_target on smithi053 to 913.0M 2024-09-19T06:25:07.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:06 smithi053 ceph-mon[27968]: pgmap v468: 129 pgs: 129 active+clean; 133 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 2.0 MiB/s wr, 100 op/s 2024-09-19T06:25:07.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:06 smithi053 ceph-mon[27968]: Adjusting osd_memory_target on smithi053 to 913.0M 2024-09-19T06:25:07.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 312228 4.07 MB/sec execute 294 sec latency 14.193 ms 2024-09-19T06:25:07.700 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1027430 13.70 MB/sec execute 291 sec latency 33.433 ms 2024-09-19T06:25:08.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 312777 4.06 MB/sec execute 295 sec latency 11.890 ms 2024-09-19T06:25:08.700 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1030966 13.72 MB/sec execute 292 sec latency 41.438 ms 2024-09-19T06:25:08.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:08 smithi155 ceph-mon[35292]: pgmap v469: 129 pgs: 129 active+clean; 135 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.8 MiB/s wr, 99 op/s 2024-09-19T06:25:09.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:08 smithi053 ceph-mon[27968]: pgmap v469: 129 pgs: 129 active+clean; 135 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.8 MiB/s wr, 99 op/s 2024-09-19T06:25:09.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 313613 4.06 MB/sec execute 296 sec latency 16.087 ms 2024-09-19T06:25:09.700 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1034459 13.74 MB/sec execute 293 sec latency 41.825 ms 2024-09-19T06:25:10.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 314463 4.05 MB/sec execute 297 sec latency 13.273 ms 2024-09-19T06:25:10.701 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1037105 13.74 MB/sec execute 294 sec latency 58.584 ms 2024-09-19T06:25:10.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:10 smithi155 ceph-mon[35292]: pgmap v470: 129 pgs: 129 active+clean; 135 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.2 MiB/s wr, 100 op/s 2024-09-19T06:25:11.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:10 smithi053 ceph-mon[27968]: pgmap v470: 129 pgs: 129 active+clean; 135 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.2 MiB/s wr, 100 op/s 2024-09-19T06:25:11.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 315439 4.06 MB/sec execute 298 sec latency 14.178 ms 2024-09-19T06:25:11.700 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1040300 13.75 MB/sec execute 295 sec latency 33.418 ms 2024-09-19T06:25:12.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 316020 4.06 MB/sec execute 299 sec latency 11.885 ms 2024-09-19T06:25:12.701 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1043252 13.75 MB/sec execute 296 sec latency 50.149 ms 2024-09-19T06:25:12.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:12 smithi155 ceph-mon[35292]: pgmap v471: 129 pgs: 129 active+clean; 135 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.7 MiB/s wr, 81 op/s 2024-09-19T06:25:13.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:12 smithi053 ceph-mon[27968]: pgmap v471: 129 pgs: 129 active+clean; 135 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.7 MiB/s wr, 81 op/s 2024-09-19T06:25:13.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 316648 4.06 MB/sec execute 300 sec latency 17.837 ms 2024-09-19T06:25:13.700 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1046307 13.76 MB/sec execute 297 sec latency 30.867 ms 2024-09-19T06:25:14.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 317516 4.06 MB/sec execute 301 sec latency 17.149 ms 2024-09-19T06:25:14.701 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1049602 13.78 MB/sec execute 298 sec latency 33.648 ms 2024-09-19T06:25:14.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:14 smithi155 ceph-mon[35292]: pgmap v472: 129 pgs: 129 active+clean; 137 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.2 MiB/s wr, 111 op/s 2024-09-19T06:25:14.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:14 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:25:15.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:14 smithi053 ceph-mon[27968]: pgmap v472: 129 pgs: 129 active+clean; 137 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.2 MiB/s wr, 111 op/s 2024-09-19T06:25:15.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:14 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:25:15.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 318304 4.05 MB/sec execute 302 sec latency 20.202 ms 2024-09-19T06:25:15.701 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1052432 13.78 MB/sec execute 299 sec latency 66.751 ms 2024-09-19T06:25:16.627 INFO:tasks.workunit.client.1.smithi155.stdout: 1 319222 4.07 MB/sec execute 303 sec latency 12.222 ms 2024-09-19T06:25:16.701 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1055865 13.80 MB/sec execute 300 sec latency 41.781 ms 2024-09-19T06:25:16.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:16 smithi053 ceph-mon[27968]: pgmap v473: 129 pgs: 129 active+clean; 137 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 2.0 MiB/s wr, 92 op/s 2024-09-19T06:25:16.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:16 smithi155 ceph-mon[35292]: pgmap v473: 129 pgs: 129 active+clean; 137 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 2.0 MiB/s wr, 92 op/s 2024-09-19T06:25:17.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 319816 4.06 MB/sec execute 304 sec latency 15.448 ms 2024-09-19T06:25:17.701 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1059401 13.82 MB/sec execute 301 sec latency 33.688 ms 2024-09-19T06:25:18.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 320537 4.06 MB/sec execute 305 sec latency 16.486 ms 2024-09-19T06:25:18.701 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1062885 13.83 MB/sec execute 302 sec latency 42.006 ms 2024-09-19T06:25:18.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:18 smithi155 ceph-mon[35292]: pgmap v474: 129 pgs: 129 active+clean; 137 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.8 MiB/s wr, 85 op/s 2024-09-19T06:25:19.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:18 smithi053 ceph-mon[27968]: pgmap v474: 129 pgs: 129 active+clean; 137 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.8 MiB/s wr, 85 op/s 2024-09-19T06:25:19.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 321328 4.05 MB/sec execute 306 sec latency 41.659 ms 2024-09-19T06:25:19.701 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1064914 13.82 MB/sec execute 303 sec latency 242.374 ms 2024-09-19T06:25:20.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 322238 4.06 MB/sec execute 307 sec latency 11.463 ms 2024-09-19T06:25:20.701 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1067328 13.82 MB/sec execute 304 sec latency 157.881 ms 2024-09-19T06:25:20.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:20 smithi155 ceph-mon[35292]: pgmap v475: 129 pgs: 129 active+clean; 138 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.0 MiB/s wr, 92 op/s 2024-09-19T06:25:21.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:20 smithi053 ceph-mon[27968]: pgmap v475: 129 pgs: 129 active+clean; 138 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.0 MiB/s wr, 92 op/s 2024-09-19T06:25:21.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 323015 4.07 MB/sec execute 308 sec latency 12.003 ms 2024-09-19T06:25:21.701 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1070051 13.82 MB/sec execute 305 sec latency 157.091 ms 2024-09-19T06:25:22.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 323550 4.06 MB/sec execute 309 sec latency 16.448 ms 2024-09-19T06:25:22.701 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1073588 13.84 MB/sec execute 306 sec latency 27.017 ms 2024-09-19T06:25:22.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:22 smithi155 ceph-mon[35292]: pgmap v476: 129 pgs: 129 active+clean; 139 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.6 MiB/s wr, 85 op/s 2024-09-19T06:25:23.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:22 smithi053 ceph-mon[27968]: pgmap v476: 129 pgs: 129 active+clean; 139 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.6 MiB/s wr, 85 op/s 2024-09-19T06:25:23.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 324381 4.06 MB/sec execute 310 sec latency 13.685 ms 2024-09-19T06:25:23.701 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1077069 13.86 MB/sec execute 307 sec latency 50.061 ms 2024-09-19T06:25:24.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 325219 4.06 MB/sec execute 311 sec latency 15.460 ms 2024-09-19T06:25:24.701 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1079249 13.85 MB/sec execute 308 sec latency 264.844 ms 2024-09-19T06:25:24.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:24 smithi155 ceph-mon[35292]: pgmap v477: 129 pgs: 129 active+clean; 140 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 5.1 KiB/s rd, 1.9 MiB/s wr, 110 op/s 2024-09-19T06:25:25.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:24 smithi053 ceph-mon[27968]: pgmap v477: 129 pgs: 129 active+clean; 140 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 5.1 KiB/s rd, 1.9 MiB/s wr, 110 op/s 2024-09-19T06:25:25.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 326105 4.07 MB/sec execute 312 sec latency 43.065 ms 2024-09-19T06:25:25.701 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1079524 13.81 MB/sec execute 309 sec latency 274.998 ms 2024-09-19T06:25:26.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 326728 4.07 MB/sec execute 313 sec latency 53.331 ms 2024-09-19T06:25:26.701 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1079665 13.77 MB/sec execute 310 sec latency 308.212 ms 2024-09-19T06:25:26.906 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:26 smithi053 ceph-mon[27968]: pgmap v478: 129 pgs: 129 active+clean; 140 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.8 MiB/s wr, 88 op/s 2024-09-19T06:25:26.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:26 smithi155 ceph-mon[35292]: pgmap v478: 129 pgs: 129 active+clean; 140 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.8 MiB/s wr, 88 op/s 2024-09-19T06:25:27.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 327380 4.06 MB/sec execute 314 sec latency 36.116 ms 2024-09-19T06:25:27.701 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1080553 13.74 MB/sec execute 311 sec latency 333.404 ms 2024-09-19T06:25:28.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:27 smithi155 ceph-mon[35292]: pgmap v479: 129 pgs: 129 active+clean; 140 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 84 op/s 2024-09-19T06:25:28.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:27 smithi053 ceph-mon[27968]: pgmap v479: 129 pgs: 129 active+clean; 140 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 84 op/s 2024-09-19T06:25:28.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 328143 4.06 MB/sec execute 315 sec latency 41.852 ms 2024-09-19T06:25:28.701 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1080604 13.70 MB/sec execute 312 sec latency 308.526 ms 2024-09-19T06:25:29.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:28 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:25:29.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:28 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:25:29.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 328844 4.05 MB/sec execute 316 sec latency 65.900 ms 2024-09-19T06:25:29.701 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1080611 13.65 MB/sec execute 313 sec latency 308.472 ms 2024-09-19T06:25:30.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:29 smithi155 ceph-mon[35292]: pgmap v480: 129 pgs: 129 active+clean; 142 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.1 MiB/s wr, 111 op/s 2024-09-19T06:25:30.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:29 smithi053 ceph-mon[27968]: pgmap v480: 129 pgs: 129 active+clean; 142 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.1 MiB/s wr, 111 op/s 2024-09-19T06:25:30.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 329671 4.06 MB/sec execute 317 sec latency 37.785 ms 2024-09-19T06:25:30.701 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1080674 13.61 MB/sec execute 314 sec latency 325.227 ms 2024-09-19T06:25:31.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 330349 4.06 MB/sec execute 318 sec latency 17.170 ms 2024-09-19T06:25:31.701 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1080681 13.57 MB/sec execute 315 sec latency 308.527 ms 2024-09-19T06:25:32.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:31 smithi155 ceph-mon[35292]: pgmap v481: 129 pgs: 129 active+clean; 142 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.6 MiB/s wr, 82 op/s 2024-09-19T06:25:32.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:31 smithi053 ceph-mon[27968]: pgmap v481: 129 pgs: 129 active+clean; 142 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.6 MiB/s wr, 82 op/s 2024-09-19T06:25:32.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 331055 4.06 MB/sec execute 319 sec latency 22.852 ms 2024-09-19T06:25:32.701 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1084097 13.58 MB/sec execute 316 sec latency 258.505 ms 2024-09-19T06:25:33.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 331931 4.06 MB/sec execute 320 sec latency 15.130 ms 2024-09-19T06:25:33.701 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1086614 13.58 MB/sec execute 317 sec latency 48.859 ms 2024-09-19T06:25:34.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:33 smithi155 ceph-mon[35292]: pgmap v482: 129 pgs: 129 active+clean; 145 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.0 MiB/s wr, 105 op/s 2024-09-19T06:25:34.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:33 smithi053 ceph-mon[27968]: pgmap v482: 129 pgs: 129 active+clean; 145 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.0 MiB/s wr, 105 op/s 2024-09-19T06:25:34.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 332826 4.06 MB/sec execute 321 sec latency 11.621 ms 2024-09-19T06:25:34.701 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1089379 13.58 MB/sec execute 318 sec latency 63.964 ms 2024-09-19T06:25:35.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 333594 4.07 MB/sec execute 322 sec latency 11.356 ms 2024-09-19T06:25:35.701 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1092148 13.59 MB/sec execute 319 sec latency 99.689 ms 2024-09-19T06:25:36.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:35 smithi155 ceph-mon[35292]: pgmap v483: 129 pgs: 129 active+clean; 143 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.9 MiB/s wr, 87 op/s 2024-09-19T06:25:36.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:35 smithi053 ceph-mon[27968]: pgmap v483: 129 pgs: 129 active+clean; 143 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.9 MiB/s wr, 87 op/s 2024-09-19T06:25:36.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 334132 4.06 MB/sec execute 323 sec latency 17.078 ms 2024-09-19T06:25:36.701 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1094937 13.59 MB/sec execute 320 sec latency 66.738 ms 2024-09-19T06:25:37.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 334920 4.06 MB/sec execute 324 sec latency 60.798 ms 2024-09-19T06:25:37.701 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1096972 13.59 MB/sec execute 321 sec latency 265.506 ms 2024-09-19T06:25:38.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:37 smithi155 ceph-mon[35292]: pgmap v484: 129 pgs: 129 active+clean; 143 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.7 MiB/s wr, 84 op/s 2024-09-19T06:25:38.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:37 smithi053 ceph-mon[27968]: pgmap v484: 129 pgs: 129 active+clean; 143 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.7 MiB/s wr, 84 op/s 2024-09-19T06:25:38.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 335730 4.06 MB/sec execute 325 sec latency 40.153 ms 2024-09-19T06:25:38.701 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1098282 13.57 MB/sec execute 322 sec latency 299.672 ms 2024-09-19T06:25:39.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 336657 4.06 MB/sec execute 326 sec latency 12.972 ms 2024-09-19T06:25:39.701 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1100796 13.57 MB/sec execute 323 sec latency 41.847 ms 2024-09-19T06:25:40.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:39 smithi155 ceph-mon[35292]: pgmap v485: 129 pgs: 129 active+clean; 143 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.1 MiB/s wr, 108 op/s 2024-09-19T06:25:40.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:39 smithi053 ceph-mon[27968]: pgmap v485: 129 pgs: 129 active+clean; 143 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.1 MiB/s wr, 108 op/s 2024-09-19T06:25:40.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 337252 4.07 MB/sec execute 327 sec latency 14.596 ms 2024-09-19T06:25:40.701 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1103910 13.58 MB/sec execute 324 sec latency 53.302 ms 2024-09-19T06:25:41.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 337844 4.06 MB/sec execute 328 sec latency 16.046 ms 2024-09-19T06:25:41.701 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1106334 13.58 MB/sec execute 325 sec latency 141.969 ms 2024-09-19T06:25:42.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:41 smithi155 ceph-mon[35292]: pgmap v486: 129 pgs: 129 active+clean; 143 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.6 MiB/s wr, 78 op/s 2024-09-19T06:25:42.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:41 smithi053 ceph-mon[27968]: pgmap v486: 129 pgs: 129 active+clean; 143 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.6 MiB/s wr, 78 op/s 2024-09-19T06:25:42.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 338646 4.06 MB/sec execute 329 sec latency 13.683 ms 2024-09-19T06:25:42.702 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1109224 13.58 MB/sec execute 326 sec latency 33.443 ms 2024-09-19T06:25:43.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 339471 4.06 MB/sec execute 330 sec latency 12.705 ms 2024-09-19T06:25:43.702 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1112668 13.60 MB/sec execute 327 sec latency 33.408 ms 2024-09-19T06:25:44.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:43 smithi155 ceph-mon[35292]: pgmap v487: 129 pgs: 129 active+clean; 145 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 5.1 KiB/s rd, 1.8 MiB/s wr, 96 op/s 2024-09-19T06:25:44.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:43 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:25:44.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:43 smithi053 ceph-mon[27968]: pgmap v487: 129 pgs: 129 active+clean; 145 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 5.1 KiB/s rd, 1.8 MiB/s wr, 96 op/s 2024-09-19T06:25:44.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:43 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:25:44.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 340298 4.06 MB/sec execute 331 sec latency 72.514 ms 2024-09-19T06:25:44.702 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1113272 13.57 MB/sec execute 328 sec latency 300.169 ms 2024-09-19T06:25:45.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 340901 4.06 MB/sec execute 332 sec latency 84.453 ms 2024-09-19T06:25:45.702 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1116005 13.57 MB/sec execute 329 sec latency 282.715 ms 2024-09-19T06:25:46.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:45 smithi155 ceph-mon[35292]: pgmap v488: 129 pgs: 129 active+clean; 145 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.8 MiB/s wr, 87 op/s 2024-09-19T06:25:46.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:45 smithi053 ceph-mon[27968]: pgmap v488: 129 pgs: 129 active+clean; 145 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.8 MiB/s wr, 87 op/s 2024-09-19T06:25:46.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 341561 4.06 MB/sec execute 333 sec latency 15.392 ms 2024-09-19T06:25:46.702 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1117674 13.55 MB/sec execute 330 sec latency 117.524 ms 2024-09-19T06:25:47.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 342389 4.06 MB/sec execute 334 sec latency 15.463 ms 2024-09-19T06:25:47.702 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1119866 13.55 MB/sec execute 331 sec latency 56.151 ms 2024-09-19T06:25:48.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:47 smithi155 ceph-mon[35292]: pgmap v489: 129 pgs: 129 active+clean; 145 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.6 MiB/s wr, 85 op/s 2024-09-19T06:25:48.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:47 smithi053 ceph-mon[27968]: pgmap v489: 129 pgs: 129 active+clean; 145 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.6 MiB/s wr, 85 op/s 2024-09-19T06:25:48.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 343137 4.05 MB/sec execute 335 sec latency 13.319 ms 2024-09-19T06:25:48.702 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1123341 13.56 MB/sec execute 332 sec latency 33.416 ms 2024-09-19T06:25:49.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 344039 4.07 MB/sec execute 336 sec latency 12.978 ms 2024-09-19T06:25:49.702 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1126711 13.58 MB/sec execute 333 sec latency 83.387 ms 2024-09-19T06:25:50.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:49 smithi155 ceph-mon[35292]: pgmap v490: 129 pgs: 129 active+clean; 147 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.1 MiB/s wr, 107 op/s 2024-09-19T06:25:50.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:49 smithi053 ceph-mon[27968]: pgmap v490: 129 pgs: 129 active+clean; 147 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.1 MiB/s wr, 107 op/s 2024-09-19T06:25:50.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 344625 4.06 MB/sec execute 337 sec latency 12.077 ms 2024-09-19T06:25:50.702 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1129301 13.58 MB/sec execute 334 sec latency 181.371 ms 2024-09-19T06:25:51.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 345350 4.06 MB/sec execute 338 sec latency 44.836 ms 2024-09-19T06:25:51.702 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1130190 13.56 MB/sec execute 335 sec latency 291.746 ms 2024-09-19T06:25:52.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:51 smithi155 ceph-mon[35292]: pgmap v491: 129 pgs: 129 active+clean; 147 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 79 op/s 2024-09-19T06:25:52.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:51 smithi053 ceph-mon[27968]: pgmap v491: 129 pgs: 129 active+clean; 147 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 79 op/s 2024-09-19T06:25:52.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 346210 4.05 MB/sec execute 339 sec latency 26.280 ms 2024-09-19T06:25:52.702 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1132542 13.55 MB/sec execute 336 sec latency 275.203 ms 2024-09-19T06:25:53.628 INFO:tasks.workunit.client.1.smithi155.stdout: 1 347058 4.06 MB/sec execute 340 sec latency 12.401 ms 2024-09-19T06:25:53.702 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1135877 13.56 MB/sec execute 337 sec latency 33.130 ms 2024-09-19T06:25:54.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:53 smithi155 ceph-mon[35292]: pgmap v492: 129 pgs: 129 active+clean; 149 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 1.9 MiB/s wr, 88 op/s 2024-09-19T06:25:54.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:53 smithi053 ceph-mon[27968]: pgmap v492: 129 pgs: 129 active+clean; 149 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 1.9 MiB/s wr, 88 op/s 2024-09-19T06:25:54.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 347838 4.07 MB/sec execute 341 sec latency 12.052 ms 2024-09-19T06:25:54.702 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1139000 13.57 MB/sec execute 338 sec latency 28.331 ms 2024-09-19T06:25:55.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 348433 4.06 MB/sec execute 342 sec latency 16.254 ms 2024-09-19T06:25:55.702 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1141839 13.58 MB/sec execute 339 sec latency 56.026 ms 2024-09-19T06:25:56.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:55 smithi053 ceph-mon[27968]: pgmap v493: 129 pgs: 129 active+clean; 148 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.6 KiB/s rd, 2.0 MiB/s wr, 96 op/s 2024-09-19T06:25:56.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:55 smithi155 ceph-mon[35292]: pgmap v493: 129 pgs: 129 active+clean; 148 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.6 KiB/s rd, 2.0 MiB/s wr, 96 op/s 2024-09-19T06:25:56.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 349235 4.06 MB/sec execute 343 sec latency 14.074 ms 2024-09-19T06:25:56.702 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1144630 13.58 MB/sec execute 340 sec latency 75.098 ms 2024-09-19T06:25:57.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 350048 4.06 MB/sec execute 344 sec latency 41.289 ms 2024-09-19T06:25:57.702 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1146663 13.58 MB/sec execute 341 sec latency 315.990 ms 2024-09-19T06:25:58.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:57 smithi053 ceph-mon[27968]: pgmap v494: 129 pgs: 129 active+clean; 148 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.7 MiB/s wr, 84 op/s 2024-09-19T06:25:58.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:57 smithi155 ceph-mon[35292]: pgmap v494: 129 pgs: 129 active+clean; 148 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.7 MiB/s wr, 84 op/s 2024-09-19T06:25:58.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 350931 4.06 MB/sec execute 345 sec latency 46.009 ms 2024-09-19T06:25:58.702 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1147046 13.54 MB/sec execute 342 sec latency 341.743 ms 2024-09-19T06:25:59.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:58 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:25:59.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:58 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:25:59.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 351604 4.07 MB/sec execute 346 sec latency 10.402 ms 2024-09-19T06:25:59.702 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1150213 13.55 MB/sec execute 343 sec latency 41.888 ms 2024-09-19T06:26:00.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:25:59 smithi053 ceph-mon[27968]: pgmap v495: 129 pgs: 129 active+clean; 150 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.2 MiB/s wr, 104 op/s 2024-09-19T06:26:00.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:25:59 smithi155 ceph-mon[35292]: pgmap v495: 129 pgs: 129 active+clean; 150 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.2 MiB/s wr, 104 op/s 2024-09-19T06:26:00.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 352324 4.06 MB/sec execute 347 sec latency 16.081 ms 2024-09-19T06:26:00.702 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1153213 13.55 MB/sec execute 344 sec latency 27.051 ms 2024-09-19T06:26:01.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 353207 4.06 MB/sec execute 348 sec latency 11.825 ms 2024-09-19T06:26:01.702 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1155311 13.55 MB/sec execute 345 sec latency 116.503 ms 2024-09-19T06:26:02.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:01 smithi053 ceph-mon[27968]: pgmap v496: 129 pgs: 129 active+clean; 149 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.6 MiB/s wr, 80 op/s 2024-09-19T06:26:02.484 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:01 smithi155 ceph-mon[35292]: pgmap v496: 129 pgs: 129 active+clean; 149 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.6 MiB/s wr, 80 op/s 2024-09-19T06:26:02.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 354122 4.07 MB/sec execute 349 sec latency 12.007 ms 2024-09-19T06:26:02.702 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1158668 13.57 MB/sec execute 346 sec latency 203.809 ms 2024-09-19T06:26:03.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 354902 4.07 MB/sec execute 350 sec latency 11.728 ms 2024-09-19T06:26:03.703 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1162163 13.58 MB/sec execute 347 sec latency 33.523 ms 2024-09-19T06:26:04.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:03 smithi053 ceph-mon[27968]: pgmap v497: 129 pgs: 129 active+clean; 149 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 1.9 MiB/s wr, 90 op/s 2024-09-19T06:26:04.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:03 smithi155 ceph-mon[35292]: pgmap v497: 129 pgs: 129 active+clean; 149 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 1.9 MiB/s wr, 90 op/s 2024-09-19T06:26:04.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 355455 4.06 MB/sec execute 351 sec latency 15.295 ms 2024-09-19T06:26:04.702 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1165699 13.60 MB/sec execute 348 sec latency 33.661 ms 2024-09-19T06:26:05.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 356258 4.07 MB/sec execute 352 sec latency 16.695 ms 2024-09-19T06:26:05.702 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1168211 13.60 MB/sec execute 349 sec latency 66.850 ms 2024-09-19T06:26:06.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:05 smithi053 ceph-mon[27968]: pgmap v498: 129 pgs: 129 active+clean; 151 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.0 MiB/s wr, 106 op/s 2024-09-19T06:26:06.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:05 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:26:06.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:05 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:26:06.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:05 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:26:06.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:05 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:26:06.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:05 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:26:06.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:05 smithi053 ceph-mon[27968]: Adjusting osd_memory_target on smithi155 to 1680M 2024-09-19T06:26:06.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:05 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:26:06.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:05 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:26:06.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:05 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:26:06.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:05 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:26:06.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:05 smithi155 ceph-mon[35292]: pgmap v498: 129 pgs: 129 active+clean; 151 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.0 MiB/s wr, 106 op/s 2024-09-19T06:26:06.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:05 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:26:06.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:05 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:26:06.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:05 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:26:06.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:05 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:26:06.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:05 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:26:06.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:05 smithi155 ceph-mon[35292]: Adjusting osd_memory_target on smithi155 to 1680M 2024-09-19T06:26:06.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:05 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:26:06.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:05 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:26:06.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:05 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:26:06.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:05 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:26:06.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 357065 4.06 MB/sec execute 353 sec latency 14.098 ms 2024-09-19T06:26:06.702 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1171046 13.60 MB/sec execute 350 sec latency 33.876 ms 2024-09-19T06:26:07.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 358009 4.07 MB/sec execute 354 sec latency 12.006 ms 2024-09-19T06:26:07.702 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1173999 13.61 MB/sec execute 351 sec latency 58.503 ms 2024-09-19T06:26:08.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:07 smithi053 ceph-mon[27968]: pgmap v499: 129 pgs: 129 active+clean; 151 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 MiB/s wr, 84 op/s 2024-09-19T06:26:08.284 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:07 smithi155 ceph-mon[35292]: pgmap v499: 129 pgs: 129 active+clean; 151 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 MiB/s wr, 84 op/s 2024-09-19T06:26:08.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 358643 4.07 MB/sec execute 355 sec latency 9.174 ms 2024-09-19T06:26:08.702 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1176650 13.61 MB/sec execute 352 sec latency 34.543 ms 2024-09-19T06:26:09.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 359305 4.06 MB/sec execute 356 sec latency 16.375 ms 2024-09-19T06:26:09.702 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1180087 13.62 MB/sec execute 353 sec latency 40.474 ms 2024-09-19T06:26:10.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:10 smithi053 ceph-mon[27968]: pgmap v500: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.2 MiB/s wr, 107 op/s 2024-09-19T06:26:10.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:10 smithi155 ceph-mon[35292]: pgmap v500: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.2 MiB/s wr, 107 op/s 2024-09-19T06:26:10.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 360154 4.06 MB/sec execute 357 sec latency 11.692 ms 2024-09-19T06:26:10.702 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1183554 13.64 MB/sec execute 354 sec latency 44.812 ms 2024-09-19T06:26:11.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 360998 4.07 MB/sec execute 358 sec latency 11.832 ms 2024-09-19T06:26:11.703 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1187030 13.65 MB/sec execute 355 sec latency 33.541 ms 2024-09-19T06:26:12.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:12 smithi053 ceph-mon[27968]: pgmap v501: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.7 MiB/s wr, 82 op/s 2024-09-19T06:26:12.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:12 smithi155 ceph-mon[35292]: pgmap v501: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.7 MiB/s wr, 82 op/s 2024-09-19T06:26:12.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 361826 4.07 MB/sec execute 359 sec latency 11.751 ms 2024-09-19T06:26:12.703 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1190523 13.67 MB/sec execute 356 sec latency 49.545 ms 2024-09-19T06:26:13.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 362418 4.07 MB/sec execute 360 sec latency 14.473 ms 2024-09-19T06:26:13.703 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1193165 13.67 MB/sec execute 357 sec latency 33.490 ms 2024-09-19T06:26:14.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:14 smithi053 ceph-mon[27968]: pgmap v502: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.0 MiB/s wr, 100 op/s 2024-09-19T06:26:14.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:14 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:26:14.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:14 smithi155 ceph-mon[35292]: pgmap v502: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.0 MiB/s wr, 100 op/s 2024-09-19T06:26:14.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:14 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:26:14.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 363264 4.07 MB/sec execute 361 sec latency 15.287 ms 2024-09-19T06:26:14.703 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1196148 13.68 MB/sec execute 358 sec latency 34.089 ms 2024-09-19T06:26:15.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 364128 4.06 MB/sec execute 362 sec latency 19.282 ms 2024-09-19T06:26:15.703 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1198824 13.68 MB/sec execute 359 sec latency 66.436 ms 2024-09-19T06:26:16.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:16 smithi053 ceph-mon[27968]: pgmap v503: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 2.1 MiB/s wr, 101 op/s 2024-09-19T06:26:16.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:16 smithi155 ceph-mon[35292]: pgmap v503: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 2.1 MiB/s wr, 101 op/s 2024-09-19T06:26:16.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 365087 4.07 MB/sec execute 363 sec latency 12.932 ms 2024-09-19T06:26:16.703 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1201960 13.69 MB/sec execute 360 sec latency 49.289 ms 2024-09-19T06:26:17.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 365694 4.08 MB/sec execute 364 sec latency 13.434 ms 2024-09-19T06:26:17.703 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1204904 13.69 MB/sec execute 361 sec latency 91.393 ms 2024-09-19T06:26:18.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:18 smithi053 ceph-mon[27968]: pgmap v504: 129 pgs: 129 active+clean; 154 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.9 MiB/s wr, 98 op/s 2024-09-19T06:26:18.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:18 smithi155 ceph-mon[35292]: pgmap v504: 129 pgs: 129 active+clean; 154 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.9 MiB/s wr, 98 op/s 2024-09-19T06:26:18.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 366304 4.07 MB/sec execute 365 sec latency 16.976 ms 2024-09-19T06:26:18.703 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1208369 13.71 MB/sec execute 362 sec latency 26.617 ms 2024-09-19T06:26:19.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 367053 4.07 MB/sec execute 366 sec latency 89.973 ms 2024-09-19T06:26:19.703 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1210899 13.71 MB/sec execute 363 sec latency 250.063 ms 2024-09-19T06:26:20.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:20 smithi053 ceph-mon[27968]: pgmap v505: 129 pgs: 129 active+clean; 155 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.3 MiB/s wr, 104 op/s 2024-09-19T06:26:20.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:20 smithi155 ceph-mon[35292]: pgmap v505: 129 pgs: 129 active+clean; 155 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.3 MiB/s wr, 104 op/s 2024-09-19T06:26:20.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 367845 4.06 MB/sec execute 367 sec latency 19.555 ms 2024-09-19T06:26:20.703 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1213538 13.70 MB/sec execute 364 sec latency 133.449 ms 2024-09-19T06:26:21.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 368680 4.07 MB/sec execute 368 sec latency 12.632 ms 2024-09-19T06:26:21.703 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1216301 13.71 MB/sec execute 365 sec latency 71.686 ms 2024-09-19T06:26:22.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:22 smithi053 ceph-mon[27968]: pgmap v506: 129 pgs: 129 active+clean; 155 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.6 MiB/s wr, 78 op/s 2024-09-19T06:26:22.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:22 smithi155 ceph-mon[35292]: pgmap v506: 129 pgs: 129 active+clean; 155 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.6 MiB/s wr, 78 op/s 2024-09-19T06:26:22.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 369369 4.07 MB/sec execute 369 sec latency 12.926 ms 2024-09-19T06:26:22.703 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1219094 13.71 MB/sec execute 366 sec latency 108.448 ms 2024-09-19T06:26:23.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 370105 4.07 MB/sec execute 370 sec latency 22.205 ms 2024-09-19T06:26:23.703 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1222556 13.73 MB/sec execute 367 sec latency 74.816 ms 2024-09-19T06:26:24.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:24 smithi053 ceph-mon[27968]: pgmap v507: 129 pgs: 129 active+clean; 157 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.0 MiB/s wr, 103 op/s 2024-09-19T06:26:24.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:24 smithi155 ceph-mon[35292]: pgmap v507: 129 pgs: 129 active+clean; 157 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.0 MiB/s wr, 103 op/s 2024-09-19T06:26:24.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 370980 4.07 MB/sec execute 371 sec latency 13.722 ms 2024-09-19T06:26:24.703 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1226040 13.74 MB/sec execute 368 sec latency 33.401 ms 2024-09-19T06:26:25.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 371874 4.07 MB/sec execute 372 sec latency 55.259 ms 2024-09-19T06:26:25.703 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1226103 13.71 MB/sec execute 369 sec latency 341.473 ms 2024-09-19T06:26:26.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:26 smithi053 ceph-mon[27968]: pgmap v508: 129 pgs: 129 active+clean; 156 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 2.0 MiB/s wr, 92 op/s 2024-09-19T06:26:26.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:26 smithi155 ceph-mon[35292]: pgmap v508: 129 pgs: 129 active+clean; 156 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 2.0 MiB/s wr, 92 op/s 2024-09-19T06:26:26.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 372558 4.08 MB/sec execute 373 sec latency 77.837 ms 2024-09-19T06:26:26.703 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1226113 13.67 MB/sec execute 370 sec latency 283.512 ms 2024-09-19T06:26:27.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 373056 4.07 MB/sec execute 374 sec latency 54.534 ms 2024-09-19T06:26:27.703 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1226224 13.63 MB/sec execute 371 sec latency 341.929 ms 2024-09-19T06:26:28.325 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:28 smithi053 ceph-mon[27968]: pgmap v509: 129 pgs: 129 active+clean; 157 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.8 MiB/s wr, 91 op/s 2024-09-19T06:26:28.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:28 smithi155 ceph-mon[35292]: pgmap v509: 129 pgs: 129 active+clean; 157 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.8 MiB/s wr, 91 op/s 2024-09-19T06:26:28.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 373893 4.07 MB/sec execute 375 sec latency 17.491 ms 2024-09-19T06:26:28.705 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1226789 13.61 MB/sec execute 372 sec latency 283.151 ms 2024-09-19T06:26:29.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:26:29.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:29 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:26:29.629 INFO:tasks.workunit.client.1.smithi155.stdout: 1 374654 4.06 MB/sec execute 376 sec latency 76.644 ms 2024-09-19T06:26:29.703 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1227193 13.57 MB/sec execute 373 sec latency 290.820 ms 2024-09-19T06:26:30.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:30 smithi053 ceph-mon[27968]: pgmap v510: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.0 MiB/s wr, 95 op/s 2024-09-19T06:26:30.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:30 smithi155 ceph-mon[35292]: pgmap v510: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.0 MiB/s wr, 95 op/s 2024-09-19T06:26:30.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 375423 4.07 MB/sec execute 377 sec latency 50.186 ms 2024-09-19T06:26:30.703 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1227679 13.54 MB/sec execute 374 sec latency 308.558 ms 2024-09-19T06:26:31.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 376119 4.07 MB/sec execute 378 sec latency 65.039 ms 2024-09-19T06:26:31.703 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1228221 13.52 MB/sec execute 375 sec latency 291.661 ms 2024-09-19T06:26:32.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:32 smithi053 ceph-mon[27968]: pgmap v511: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 85 op/s 2024-09-19T06:26:32.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:32 smithi155 ceph-mon[35292]: pgmap v511: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 85 op/s 2024-09-19T06:26:32.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 376623 4.07 MB/sec execute 379 sec latency 42.645 ms 2024-09-19T06:26:32.703 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1229585 13.50 MB/sec execute 376 sec latency 307.550 ms 2024-09-19T06:26:33.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 377459 4.07 MB/sec execute 380 sec latency 11.675 ms 2024-09-19T06:26:33.703 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1233074 13.52 MB/sec execute 377 sec latency 103.129 ms 2024-09-19T06:26:34.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:34 smithi053 ceph-mon[27968]: pgmap v512: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 1.8 MiB/s wr, 93 op/s 2024-09-19T06:26:34.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:34 smithi155 ceph-mon[35292]: pgmap v512: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 1.8 MiB/s wr, 93 op/s 2024-09-19T06:26:34.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 378349 4.06 MB/sec execute 381 sec latency 20.173 ms 2024-09-19T06:26:34.703 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1235685 13.52 MB/sec execute 378 sec latency 33.539 ms 2024-09-19T06:26:35.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 379306 4.07 MB/sec execute 382 sec latency 16.053 ms 2024-09-19T06:26:35.703 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1237828 13.51 MB/sec execute 379 sec latency 74.712 ms 2024-09-19T06:26:36.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:36 smithi053 ceph-mon[27968]: pgmap v513: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.8 MiB/s wr, 90 op/s 2024-09-19T06:26:36.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:36 smithi155 ceph-mon[35292]: pgmap v513: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.8 MiB/s wr, 90 op/s 2024-09-19T06:26:36.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 379916 4.07 MB/sec execute 383 sec latency 13.271 ms 2024-09-19T06:26:36.703 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1240472 13.51 MB/sec execute 380 sec latency 34.945 ms 2024-09-19T06:26:37.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 380549 4.06 MB/sec execute 384 sec latency 24.189 ms 2024-09-19T06:26:37.704 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1243828 13.53 MB/sec execute 381 sec latency 91.659 ms 2024-09-19T06:26:38.284 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:38 smithi155 ceph-mon[35292]: pgmap v514: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.6 MiB/s wr, 86 op/s 2024-09-19T06:26:38.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:38 smithi053 ceph-mon[27968]: pgmap v514: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.6 MiB/s wr, 86 op/s 2024-09-19T06:26:38.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 381425 4.07 MB/sec execute 385 sec latency 15.094 ms 2024-09-19T06:26:38.704 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1247311 13.54 MB/sec execute 382 sec latency 52.913 ms 2024-09-19T06:26:39.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 382181 4.06 MB/sec execute 386 sec latency 12.394 ms 2024-09-19T06:26:39.704 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1250800 13.55 MB/sec execute 383 sec latency 33.390 ms 2024-09-19T06:26:40.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:40 smithi053 ceph-mon[27968]: pgmap v515: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 2.1 MiB/s wr, 105 op/s 2024-09-19T06:26:40.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:40 smithi155 ceph-mon[35292]: pgmap v515: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 2.1 MiB/s wr, 105 op/s 2024-09-19T06:26:40.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 382968 4.07 MB/sec execute 387 sec latency 14.653 ms 2024-09-19T06:26:40.704 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1253452 13.56 MB/sec execute 384 sec latency 70.042 ms 2024-09-19T06:26:41.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 383653 4.07 MB/sec execute 388 sec latency 13.600 ms 2024-09-19T06:26:41.704 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1256859 13.57 MB/sec execute 385 sec latency 33.637 ms 2024-09-19T06:26:42.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:42 smithi053 ceph-mon[27968]: pgmap v516: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.5 MiB/s wr, 79 op/s 2024-09-19T06:26:42.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:42 smithi155 ceph-mon[35292]: pgmap v516: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.5 MiB/s wr, 79 op/s 2024-09-19T06:26:42.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 384395 4.06 MB/sec execute 389 sec latency 15.984 ms 2024-09-19T06:26:42.704 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1259685 13.57 MB/sec execute 386 sec latency 33.697 ms 2024-09-19T06:26:43.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 385313 4.06 MB/sec execute 390 sec latency 15.880 ms 2024-09-19T06:26:43.704 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1262647 13.58 MB/sec execute 387 sec latency 40.578 ms 2024-09-19T06:26:44.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:44 smithi053 ceph-mon[27968]: pgmap v517: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 1.9 MiB/s wr, 105 op/s 2024-09-19T06:26:44.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:44 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:26:44.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:44 smithi155 ceph-mon[35292]: pgmap v517: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 1.9 MiB/s wr, 105 op/s 2024-09-19T06:26:44.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:44 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:26:44.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 386335 4.07 MB/sec execute 391 sec latency 13.606 ms 2024-09-19T06:26:44.704 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1265288 13.58 MB/sec execute 388 sec latency 51.480 ms 2024-09-19T06:26:45.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 386949 4.08 MB/sec execute 392 sec latency 12.098 ms 2024-09-19T06:26:45.704 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1268729 13.59 MB/sec execute 389 sec latency 33.701 ms 2024-09-19T06:26:46.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:46 smithi053 ceph-mon[27968]: pgmap v518: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 2.0 MiB/s wr, 104 op/s 2024-09-19T06:26:46.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:46 smithi155 ceph-mon[35292]: pgmap v518: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 2.0 MiB/s wr, 104 op/s 2024-09-19T06:26:46.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 387560 4.07 MB/sec execute 393 sec latency 20.497 ms 2024-09-19T06:26:46.704 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1272191 13.61 MB/sec execute 390 sec latency 41.308 ms 2024-09-19T06:26:47.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 388389 4.07 MB/sec execute 394 sec latency 12.302 ms 2024-09-19T06:26:47.704 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1275672 13.62 MB/sec execute 391 sec latency 50.055 ms 2024-09-19T06:26:48.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:48 smithi053 ceph-mon[27968]: pgmap v519: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.7 MiB/s wr, 84 op/s 2024-09-19T06:26:48.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:48 smithi155 ceph-mon[35292]: pgmap v519: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.7 MiB/s wr, 84 op/s 2024-09-19T06:26:48.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 389224 4.07 MB/sec execute 395 sec latency 21.133 ms 2024-09-19T06:26:48.704 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1279012 13.63 MB/sec execute 392 sec latency 116.478 ms 2024-09-19T06:26:49.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 390064 4.07 MB/sec execute 396 sec latency 9.570 ms 2024-09-19T06:26:49.704 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1281774 13.63 MB/sec execute 393 sec latency 25.169 ms 2024-09-19T06:26:50.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:50 smithi053 ceph-mon[27968]: pgmap v520: 129 pgs: 129 active+clean; 163 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.3 MiB/s wr, 109 op/s 2024-09-19T06:26:50.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:50 smithi155 ceph-mon[35292]: pgmap v520: 129 pgs: 129 active+clean; 163 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.3 MiB/s wr, 109 op/s 2024-09-19T06:26:50.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 390729 4.07 MB/sec execute 397 sec latency 14.053 ms 2024-09-19T06:26:50.704 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1284947 13.64 MB/sec execute 394 sec latency 33.316 ms 2024-09-19T06:26:51.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 391482 4.07 MB/sec execute 398 sec latency 14.078 ms 2024-09-19T06:26:51.704 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1287211 13.64 MB/sec execute 395 sec latency 92.732 ms 2024-09-19T06:26:52.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:52 smithi053 ceph-mon[27968]: pgmap v521: 129 pgs: 129 active+clean; 164 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.6 MiB/s wr, 80 op/s 2024-09-19T06:26:52.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:52 smithi155 ceph-mon[35292]: pgmap v521: 129 pgs: 129 active+clean; 164 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.6 MiB/s wr, 80 op/s 2024-09-19T06:26:52.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 392363 4.07 MB/sec execute 399 sec latency 19.501 ms 2024-09-19T06:26:52.704 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1290109 13.64 MB/sec execute 396 sec latency 33.669 ms 2024-09-19T06:26:53.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 393247 4.08 MB/sec execute 400 sec latency 12.384 ms 2024-09-19T06:26:53.704 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1293549 13.66 MB/sec execute 397 sec latency 33.674 ms 2024-09-19T06:26:54.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:54 smithi053 ceph-mon[27968]: pgmap v522: 129 pgs: 129 active+clean; 165 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 5.4 KiB/s rd, 2.0 MiB/s wr, 106 op/s 2024-09-19T06:26:54.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:54 smithi155 ceph-mon[35292]: pgmap v522: 129 pgs: 129 active+clean; 165 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 5.4 KiB/s rd, 2.0 MiB/s wr, 106 op/s 2024-09-19T06:26:54.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 394034 4.08 MB/sec execute 401 sec latency 9.516 ms 2024-09-19T06:26:54.704 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1296992 13.67 MB/sec execute 398 sec latency 116.852 ms 2024-09-19T06:26:55.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 394631 4.07 MB/sec execute 402 sec latency 20.191 ms 2024-09-19T06:26:55.704 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1300527 13.68 MB/sec execute 399 sec latency 58.429 ms 2024-09-19T06:26:56.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:56 smithi053 ceph-mon[27968]: pgmap v523: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.7 KiB/s rd, 2.0 MiB/s wr, 94 op/s 2024-09-19T06:26:56.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:56 smithi155 ceph-mon[35292]: pgmap v523: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.7 KiB/s rd, 2.0 MiB/s wr, 94 op/s 2024-09-19T06:26:56.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 395417 4.07 MB/sec execute 403 sec latency 12.548 ms 2024-09-19T06:26:56.704 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1304048 13.70 MB/sec execute 400 sec latency 41.035 ms 2024-09-19T06:26:57.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 396239 4.07 MB/sec execute 404 sec latency 12.494 ms 2024-09-19T06:26:57.704 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1306700 13.70 MB/sec execute 401 sec latency 41.643 ms 2024-09-19T06:26:58.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:58 smithi053 ceph-mon[27968]: pgmap v524: 129 pgs: 129 active+clean; 165 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.7 MiB/s wr, 88 op/s 2024-09-19T06:26:58.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:58 smithi155 ceph-mon[35292]: pgmap v524: 129 pgs: 129 active+clean; 165 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.7 MiB/s wr, 88 op/s 2024-09-19T06:26:58.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 397084 4.08 MB/sec execute 405 sec latency 73.354 ms 2024-09-19T06:26:58.704 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1307834 13.68 MB/sec execute 402 sec latency 291.876 ms 2024-09-19T06:26:59.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:26:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:26:59.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:26:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:26:59.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 397740 4.08 MB/sec execute 406 sec latency 11.578 ms 2024-09-19T06:26:59.704 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1311338 13.69 MB/sec execute 403 sec latency 33.701 ms 2024-09-19T06:27:00.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:00 smithi053 ceph-mon[27968]: pgmap v525: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 5.4 KiB/s rd, 2.1 MiB/s wr, 99 op/s 2024-09-19T06:27:00.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:00 smithi155 ceph-mon[35292]: pgmap v525: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 5.4 KiB/s rd, 2.1 MiB/s wr, 99 op/s 2024-09-19T06:27:00.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 398417 4.07 MB/sec execute 407 sec latency 38.894 ms 2024-09-19T06:27:00.704 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1314803 13.70 MB/sec execute 404 sec latency 33.679 ms 2024-09-19T06:27:01.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 399236 4.07 MB/sec execute 408 sec latency 16.862 ms 2024-09-19T06:27:01.705 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1318288 13.72 MB/sec execute 405 sec latency 33.433 ms 2024-09-19T06:27:02.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:02 smithi053 ceph-mon[27968]: pgmap v526: 129 pgs: 129 active+clean; 166 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.6 MiB/s wr, 87 op/s 2024-09-19T06:27:02.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:02 smithi155 ceph-mon[35292]: pgmap v526: 129 pgs: 129 active+clean; 166 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.6 MiB/s wr, 87 op/s 2024-09-19T06:27:02.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 400181 4.08 MB/sec execute 409 sec latency 11.766 ms 2024-09-19T06:27:02.705 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1321821 13.73 MB/sec execute 406 sec latency 42.059 ms 2024-09-19T06:27:03.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 400970 4.08 MB/sec execute 410 sec latency 11.865 ms 2024-09-19T06:27:03.705 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1324383 13.73 MB/sec execute 407 sec latency 108.008 ms 2024-09-19T06:27:04.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:04 smithi053 ceph-mon[27968]: pgmap v527: 129 pgs: 129 active+clean; 166 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 5.4 KiB/s rd, 1.8 MiB/s wr, 94 op/s 2024-09-19T06:27:04.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:04 smithi155 ceph-mon[35292]: pgmap v527: 129 pgs: 129 active+clean; 166 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 5.4 KiB/s rd, 1.8 MiB/s wr, 94 op/s 2024-09-19T06:27:04.630 INFO:tasks.workunit.client.1.smithi155.stdout: 1 401444 4.07 MB/sec execute 411 sec latency 54.327 ms 2024-09-19T06:27:04.705 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1324425 13.70 MB/sec execute 408 sec latency 299.879 ms 2024-09-19T06:27:05.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 402149 4.07 MB/sec execute 412 sec latency 39.756 ms 2024-09-19T06:27:05.705 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1325320 13.67 MB/sec execute 409 sec latency 322.740 ms 2024-09-19T06:27:06.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:06 smithi053 ceph-mon[27968]: pgmap v528: 129 pgs: 129 active+clean; 168 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 2.0 MiB/s wr, 94 op/s 2024-09-19T06:27:06.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:06 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:27:06.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:06 smithi155 ceph-mon[35292]: pgmap v528: 129 pgs: 129 active+clean; 168 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 2.0 MiB/s wr, 94 op/s 2024-09-19T06:27:06.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:06 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:27:06.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 402785 4.07 MB/sec execute 413 sec latency 86.869 ms 2024-09-19T06:27:06.705 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1325369 13.64 MB/sec execute 410 sec latency 291.886 ms 2024-09-19T06:27:07.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 403292 4.06 MB/sec execute 414 sec latency 94.423 ms 2024-09-19T06:27:07.705 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1325374 13.61 MB/sec execute 411 sec latency 446.086 ms 2024-09-19T06:27:08.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:08 smithi053 ceph-mon[27968]: pgmap v529: 129 pgs: 129 active+clean; 168 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.7 MiB/s wr, 85 op/s 2024-09-19T06:27:08.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:08 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:27:08.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:08 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:27:08.384 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:08 smithi155 ceph-mon[35292]: pgmap v529: 129 pgs: 129 active+clean; 168 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.7 MiB/s wr, 85 op/s 2024-09-19T06:27:08.384 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:08 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:27:08.385 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:08 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:27:08.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 404031 4.07 MB/sec execute 415 sec latency 177.735 ms 2024-09-19T06:27:08.705 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1325381 13.58 MB/sec execute 412 sec latency 491.311 ms 2024-09-19T06:27:09.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 404496 4.07 MB/sec execute 416 sec latency 83.380 ms 2024-09-19T06:27:09.705 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1325439 13.54 MB/sec execute 413 sec latency 441.749 ms 2024-09-19T06:27:10.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:10 smithi053 ceph-mon[27968]: pgmap v530: 129 pgs: 129 active+clean; 168 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.1 MiB/s wr, 92 op/s 2024-09-19T06:27:10.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:10 smithi155 ceph-mon[35292]: pgmap v530: 129 pgs: 129 active+clean; 168 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.1 MiB/s wr, 92 op/s 2024-09-19T06:27:10.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 404872 4.06 MB/sec execute 417 sec latency 100.829 ms 2024-09-19T06:27:10.705 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1325446 13.51 MB/sec execute 414 sec latency 432.820 ms 2024-09-19T06:27:11.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 405374 4.05 MB/sec execute 418 sec latency 89.513 ms 2024-09-19T06:27:11.705 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1325520 13.48 MB/sec execute 415 sec latency 408.527 ms 2024-09-19T06:27:12.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:12 smithi053 ceph-mon[27968]: pgmap v531: 129 pgs: 129 active+clean; 169 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.7 MiB/s wr, 89 op/s 2024-09-19T06:27:12.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:12 smithi155 ceph-mon[35292]: pgmap v531: 129 pgs: 129 active+clean; 169 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.7 MiB/s wr, 89 op/s 2024-09-19T06:27:12.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 405916 4.05 MB/sec execute 419 sec latency 107.948 ms 2024-09-19T06:27:12.705 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1325563 13.45 MB/sec execute 416 sec latency 416.982 ms 2024-09-19T06:27:13.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 406763 4.05 MB/sec execute 420 sec latency 18.098 ms 2024-09-19T06:27:13.705 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1328863 13.46 MB/sec execute 417 sec latency 192.379 ms 2024-09-19T06:27:14.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:14 smithi053 ceph-mon[27968]: pgmap v532: 129 pgs: 129 active+clean; 169 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 4.2 KiB/s rd, 1.7 MiB/s wr, 77 op/s 2024-09-19T06:27:14.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:14 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:27:14.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:14 smithi155 ceph-mon[35292]: pgmap v532: 129 pgs: 129 active+clean; 169 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 4.2 KiB/s rd, 1.7 MiB/s wr, 77 op/s 2024-09-19T06:27:14.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:14 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:27:14.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 407696 4.06 MB/sec execute 421 sec latency 17.933 ms 2024-09-19T06:27:14.705 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1329819 13.44 MB/sec execute 418 sec latency 175.357 ms 2024-09-19T06:27:15.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 408310 4.06 MB/sec execute 422 sec latency 12.411 ms 2024-09-19T06:27:15.705 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1332414 13.44 MB/sec execute 419 sec latency 80.946 ms 2024-09-19T06:27:16.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:16 smithi053 ceph-mon[27968]: pgmap v533: 129 pgs: 129 active+clean; 170 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.7 MiB/s wr, 88 op/s 2024-09-19T06:27:16.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:16 smithi155 ceph-mon[35292]: pgmap v533: 129 pgs: 129 active+clean; 170 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.7 MiB/s wr, 88 op/s 2024-09-19T06:27:16.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 408907 4.05 MB/sec execute 423 sec latency 20.821 ms 2024-09-19T06:27:16.705 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1333618 13.42 MB/sec execute 420 sec latency 85.229 ms 2024-09-19T06:27:17.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 409792 4.05 MB/sec execute 424 sec latency 10.420 ms 2024-09-19T06:27:17.705 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1336006 13.42 MB/sec execute 421 sec latency 58.109 ms 2024-09-19T06:27:18.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:18 smithi053 ceph-mon[27968]: pgmap v534: 129 pgs: 129 active+clean; 170 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.3 MiB/s wr, 65 op/s 2024-09-19T06:27:18.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:18 smithi155 ceph-mon[35292]: pgmap v534: 129 pgs: 129 active+clean; 170 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.3 MiB/s wr, 65 op/s 2024-09-19T06:27:18.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 410529 4.05 MB/sec execute 425 sec latency 40.564 ms 2024-09-19T06:27:18.705 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1336081 13.39 MB/sec execute 422 sec latency 283.511 ms 2024-09-19T06:27:19.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 411277 4.05 MB/sec execute 426 sec latency 61.073 ms 2024-09-19T06:27:19.705 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1336759 13.37 MB/sec execute 423 sec latency 346.584 ms 2024-09-19T06:27:19.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:19 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:27:19.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:19 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:27:19.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:19 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:27:19.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:19 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:27:19.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:19 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:27:19.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:19 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:27:19.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:19 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:27:19.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:19 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:27:19.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:19 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:27:19.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:19 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:27:20.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 411940 4.06 MB/sec execute 427 sec latency 13.980 ms 2024-09-19T06:27:20.705 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1339743 13.37 MB/sec execute 424 sec latency 33.195 ms 2024-09-19T06:27:20.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:20 smithi155 ceph-mon[35292]: pgmap v535: 129 pgs: 129 active+clean; 172 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.9 KiB/s rd, 1.8 MiB/s wr, 89 op/s 2024-09-19T06:27:21.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:20 smithi053 ceph-mon[27968]: pgmap v535: 129 pgs: 129 active+clean; 172 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.9 KiB/s rd, 1.8 MiB/s wr, 89 op/s 2024-09-19T06:27:21.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 412633 4.05 MB/sec execute 428 sec latency 20.214 ms 2024-09-19T06:27:21.705 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1343103 13.38 MB/sec execute 425 sec latency 164.126 ms 2024-09-19T06:27:22.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 413506 4.05 MB/sec execute 429 sec latency 12.279 ms 2024-09-19T06:27:22.705 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1346596 13.40 MB/sec execute 426 sec latency 183.448 ms 2024-09-19T06:27:22.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:22 smithi155 ceph-mon[35292]: pgmap v536: 129 pgs: 129 active+clean; 172 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.4 MiB/s wr, 80 op/s 2024-09-19T06:27:23.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:22 smithi053 ceph-mon[27968]: pgmap v536: 129 pgs: 129 active+clean; 172 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.4 MiB/s wr, 80 op/s 2024-09-19T06:27:23.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 414421 4.06 MB/sec execute 430 sec latency 19.817 ms 2024-09-19T06:27:23.705 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1350130 13.41 MB/sec execute 427 sec latency 49.242 ms 2024-09-19T06:27:24.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 415154 4.06 MB/sec execute 431 sec latency 23.059 ms 2024-09-19T06:27:24.705 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1352647 13.41 MB/sec execute 428 sec latency 33.665 ms 2024-09-19T06:27:24.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:24 smithi155 ceph-mon[35292]: pgmap v537: 129 pgs: 129 active+clean; 171 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 4.2 KiB/s rd, 1.5 MiB/s wr, 81 op/s 2024-09-19T06:27:25.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:24 smithi053 ceph-mon[27968]: pgmap v537: 129 pgs: 129 active+clean; 171 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 4.2 KiB/s rd, 1.5 MiB/s wr, 81 op/s 2024-09-19T06:27:25.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 415731 4.05 MB/sec execute 432 sec latency 16.108 ms 2024-09-19T06:27:25.705 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1355419 13.41 MB/sec execute 429 sec latency 83.415 ms 2024-09-19T06:27:26.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 416570 4.06 MB/sec execute 433 sec latency 10.402 ms 2024-09-19T06:27:26.705 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1358433 13.42 MB/sec execute 430 sec latency 41.474 ms 2024-09-19T06:27:26.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:26 smithi155 ceph-mon[35292]: pgmap v538: 129 pgs: 129 active+clean; 174 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.8 MiB/s wr, 97 op/s 2024-09-19T06:27:27.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:26 smithi053 ceph-mon[27968]: pgmap v538: 129 pgs: 129 active+clean; 174 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.8 MiB/s wr, 97 op/s 2024-09-19T06:27:27.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 417404 4.05 MB/sec execute 434 sec latency 12.518 ms 2024-09-19T06:27:27.705 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1361082 13.42 MB/sec execute 431 sec latency 33.715 ms 2024-09-19T06:27:28.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 418350 4.06 MB/sec execute 435 sec latency 13.484 ms 2024-09-19T06:27:28.705 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1364521 13.43 MB/sec execute 432 sec latency 33.491 ms 2024-09-19T06:27:28.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:28 smithi155 ceph-mon[35292]: pgmap v539: 129 pgs: 129 active+clean; 174 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.7 MiB/s wr, 83 op/s 2024-09-19T06:27:29.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:28 smithi053 ceph-mon[27968]: pgmap v539: 129 pgs: 129 active+clean; 174 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.7 MiB/s wr, 83 op/s 2024-09-19T06:27:29.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 418954 4.06 MB/sec execute 436 sec latency 14.237 ms 2024-09-19T06:27:29.705 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1367990 13.45 MB/sec execute 433 sec latency 35.436 ms 2024-09-19T06:27:29.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:29 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:27:30.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:27:30.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 419620 4.06 MB/sec execute 437 sec latency 18.771 ms 2024-09-19T06:27:30.705 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1371467 13.46 MB/sec execute 434 sec latency 50.110 ms 2024-09-19T06:27:30.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:30 smithi155 ceph-mon[35292]: pgmap v540: 129 pgs: 129 active+clean; 173 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 2.2 MiB/s wr, 107 op/s 2024-09-19T06:27:31.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:30 smithi053 ceph-mon[27968]: pgmap v540: 129 pgs: 129 active+clean; 173 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 2.2 MiB/s wr, 107 op/s 2024-09-19T06:27:31.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 420453 4.05 MB/sec execute 438 sec latency 13.688 ms 2024-09-19T06:27:31.706 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1375698 13.52 MB/sec execute 435 sec latency 27.223 ms 2024-09-19T06:27:32.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 421300 4.06 MB/sec execute 439 sec latency 18.821 ms 2024-09-19T06:27:32.706 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1378329 13.52 MB/sec execute 436 sec latency 33.452 ms 2024-09-19T06:27:32.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:32 smithi155 ceph-mon[35292]: pgmap v541: 129 pgs: 129 active+clean; 174 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 78 op/s 2024-09-19T06:27:33.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:32 smithi053 ceph-mon[27968]: pgmap v541: 129 pgs: 129 active+clean; 174 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 78 op/s 2024-09-19T06:27:33.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 422139 4.06 MB/sec execute 440 sec latency 12.257 ms 2024-09-19T06:27:33.706 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1381069 13.52 MB/sec execute 437 sec latency 47.699 ms 2024-09-19T06:27:34.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 422706 4.06 MB/sec execute 441 sec latency 12.006 ms 2024-09-19T06:27:34.706 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1383772 13.53 MB/sec execute 438 sec latency 125.215 ms 2024-09-19T06:27:34.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:34 smithi155 ceph-mon[35292]: pgmap v542: 129 pgs: 129 active+clean; 175 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.0 MiB/s wr, 104 op/s 2024-09-19T06:27:35.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:34 smithi053 ceph-mon[27968]: pgmap v542: 129 pgs: 129 active+clean; 175 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.0 MiB/s wr, 104 op/s 2024-09-19T06:27:35.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 423457 4.06 MB/sec execute 442 sec latency 23.570 ms 2024-09-19T06:27:35.706 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1386493 13.53 MB/sec execute 439 sec latency 34.657 ms 2024-09-19T06:27:36.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 424317 4.05 MB/sec execute 443 sec latency 11.810 ms 2024-09-19T06:27:36.706 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1390030 13.54 MB/sec execute 440 sec latency 47.081 ms 2024-09-19T06:27:36.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:36 smithi155 ceph-mon[35292]: pgmap v543: 129 pgs: 129 active+clean; 176 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.6 KiB/s rd, 2.1 MiB/s wr, 101 op/s 2024-09-19T06:27:37.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:36 smithi053 ceph-mon[27968]: pgmap v543: 129 pgs: 129 active+clean; 176 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.6 KiB/s rd, 2.1 MiB/s wr, 101 op/s 2024-09-19T06:27:37.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 425389 4.06 MB/sec execute 444 sec latency 16.233 ms 2024-09-19T06:27:37.706 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1393464 13.55 MB/sec execute 441 sec latency 48.830 ms 2024-09-19T06:27:38.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 425957 4.06 MB/sec execute 445 sec latency 13.140 ms 2024-09-19T06:27:38.706 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1396116 13.56 MB/sec execute 442 sec latency 41.746 ms 2024-09-19T06:27:38.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:38 smithi155 ceph-mon[35292]: pgmap v544: 129 pgs: 129 active+clean; 176 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.7 MiB/s wr, 83 op/s 2024-09-19T06:27:39.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:38 smithi053 ceph-mon[27968]: pgmap v544: 129 pgs: 129 active+clean; 176 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.7 MiB/s wr, 83 op/s 2024-09-19T06:27:39.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 426538 4.06 MB/sec execute 446 sec latency 15.883 ms 2024-09-19T06:27:39.706 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1398835 13.55 MB/sec execute 443 sec latency 107.103 ms 2024-09-19T06:27:40.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 427323 4.06 MB/sec execute 447 sec latency 24.658 ms 2024-09-19T06:27:40.706 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1400709 13.55 MB/sec execute 444 sec latency 208.527 ms 2024-09-19T06:27:40.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:40 smithi155 ceph-mon[35292]: pgmap v545: 129 pgs: 129 active+clean; 177 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 5.5 KiB/s rd, 2.2 MiB/s wr, 107 op/s 2024-09-19T06:27:41.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:40 smithi053 ceph-mon[27968]: pgmap v545: 129 pgs: 129 active+clean; 177 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 5.5 KiB/s rd, 2.2 MiB/s wr, 107 op/s 2024-09-19T06:27:41.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 428167 4.05 MB/sec execute 448 sec latency 11.810 ms 2024-09-19T06:27:41.706 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1404190 13.56 MB/sec execute 445 sec latency 33.448 ms 2024-09-19T06:27:42.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 429014 4.06 MB/sec execute 449 sec latency 22.143 ms 2024-09-19T06:27:42.706 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1407729 13.57 MB/sec execute 446 sec latency 33.462 ms 2024-09-19T06:27:42.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:42 smithi155 ceph-mon[35292]: pgmap v546: 129 pgs: 129 active+clean; 177 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.8 KiB/s rd, 1.7 MiB/s wr, 79 op/s 2024-09-19T06:27:43.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:42 smithi053 ceph-mon[27968]: pgmap v546: 129 pgs: 129 active+clean; 177 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.8 KiB/s rd, 1.7 MiB/s wr, 79 op/s 2024-09-19T06:27:43.631 INFO:tasks.workunit.client.1.smithi155.stdout: 1 429681 4.06 MB/sec execute 450 sec latency 12.011 ms 2024-09-19T06:27:43.706 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1411221 13.59 MB/sec execute 447 sec latency 33.926 ms 2024-09-19T06:27:44.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 430399 4.06 MB/sec execute 451 sec latency 15.310 ms 2024-09-19T06:27:44.706 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1413458 13.58 MB/sec execute 448 sec latency 152.740 ms 2024-09-19T06:27:44.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:44 smithi155 ceph-mon[35292]: pgmap v547: 129 pgs: 129 active+clean; 178 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 5.5 KiB/s rd, 1.9 MiB/s wr, 105 op/s 2024-09-19T06:27:44.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:44 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:27:45.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:44 smithi053 ceph-mon[27968]: pgmap v547: 129 pgs: 129 active+clean; 178 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 5.5 KiB/s rd, 1.9 MiB/s wr, 105 op/s 2024-09-19T06:27:45.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:44 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:27:45.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 431268 4.06 MB/sec execute 452 sec latency 12.639 ms 2024-09-19T06:27:45.706 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1415981 13.58 MB/sec execute 449 sec latency 123.749 ms 2024-09-19T06:27:46.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 431995 4.06 MB/sec execute 453 sec latency 78.055 ms 2024-09-19T06:27:46.706 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1416467 13.55 MB/sec execute 450 sec latency 312.554 ms 2024-09-19T06:27:46.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:46 smithi155 ceph-mon[35292]: pgmap v548: 129 pgs: 129 active+clean; 178 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.6 KiB/s rd, 2.0 MiB/s wr, 86 op/s 2024-09-19T06:27:47.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:46 smithi053 ceph-mon[27968]: pgmap v548: 129 pgs: 129 active+clean; 178 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.6 KiB/s rd, 2.0 MiB/s wr, 86 op/s 2024-09-19T06:27:47.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 432828 4.06 MB/sec execute 454 sec latency 43.856 ms 2024-09-19T06:27:47.706 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1416991 13.53 MB/sec execute 451 sec latency 314.612 ms 2024-09-19T06:27:48.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 433284 4.06 MB/sec execute 455 sec latency 87.467 ms 2024-09-19T06:27:48.706 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1417283 13.51 MB/sec execute 452 sec latency 366.805 ms 2024-09-19T06:27:48.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:48 smithi155 ceph-mon[35292]: pgmap v549: 129 pgs: 129 active+clean; 179 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.8 MiB/s wr, 98 op/s 2024-09-19T06:27:49.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:48 smithi053 ceph-mon[27968]: pgmap v549: 129 pgs: 129 active+clean; 179 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.8 MiB/s wr, 98 op/s 2024-09-19T06:27:49.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 433993 4.05 MB/sec execute 456 sec latency 38.834 ms 2024-09-19T06:27:49.706 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1417424 13.48 MB/sec execute 453 sec latency 299.150 ms 2024-09-19T06:27:50.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 434865 4.05 MB/sec execute 457 sec latency 12.496 ms 2024-09-19T06:27:50.706 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1418311 13.46 MB/sec execute 454 sec latency 266.821 ms 2024-09-19T06:27:50.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:50 smithi155 ceph-mon[35292]: pgmap v550: 129 pgs: 129 active+clean; 179 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.2 MiB/s wr, 105 op/s 2024-09-19T06:27:51.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:50 smithi053 ceph-mon[27968]: pgmap v550: 129 pgs: 129 active+clean; 179 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.2 MiB/s wr, 105 op/s 2024-09-19T06:27:51.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 435774 4.06 MB/sec execute 458 sec latency 37.570 ms 2024-09-19T06:27:51.708 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1418362 13.43 MB/sec execute 455 sec latency 300.170 ms 2024-09-19T06:27:52.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 436450 4.06 MB/sec execute 459 sec latency 82.345 ms 2024-09-19T06:27:52.706 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1418371 13.40 MB/sec execute 456 sec latency 316.807 ms 2024-09-19T06:27:52.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:52 smithi155 ceph-mon[35292]: pgmap v551: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.7 MiB/s wr, 76 op/s 2024-09-19T06:27:53.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:52 smithi053 ceph-mon[27968]: pgmap v551: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.7 MiB/s wr, 76 op/s 2024-09-19T06:27:53.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 437005 4.06 MB/sec execute 460 sec latency 26.642 ms 2024-09-19T06:27:53.706 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1419523 13.38 MB/sec execute 457 sec latency 291.866 ms 2024-09-19T06:27:54.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 437843 4.06 MB/sec execute 461 sec latency 16.579 ms 2024-09-19T06:27:54.706 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1422817 13.40 MB/sec execute 458 sec latency 33.319 ms 2024-09-19T06:27:54.984 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:54 smithi155 ceph-mon[35292]: pgmap v552: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.0 MiB/s wr, 105 op/s 2024-09-19T06:27:55.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:54 smithi053 ceph-mon[27968]: pgmap v552: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.0 MiB/s wr, 105 op/s 2024-09-19T06:27:55.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 438706 4.05 MB/sec execute 462 sec latency 16.016 ms 2024-09-19T06:27:55.706 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1425528 13.40 MB/sec execute 459 sec latency 133.500 ms 2024-09-19T06:27:56.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 439627 4.06 MB/sec execute 463 sec latency 12.888 ms 2024-09-19T06:27:56.707 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1429008 13.41 MB/sec execute 460 sec latency 99.986 ms 2024-09-19T06:27:56.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:56 smithi155 ceph-mon[35292]: pgmap v553: 129 pgs: 129 active+clean; 181 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 2.0 MiB/s wr, 85 op/s 2024-09-19T06:27:57.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:56 smithi053 ceph-mon[27968]: pgmap v553: 129 pgs: 129 active+clean; 181 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 2.0 MiB/s wr, 85 op/s 2024-09-19T06:27:57.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 440275 4.06 MB/sec execute 464 sec latency 12.444 ms 2024-09-19T06:27:57.707 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1432501 13.42 MB/sec execute 461 sec latency 33.479 ms 2024-09-19T06:27:58.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 440959 4.06 MB/sec execute 465 sec latency 16.072 ms 2024-09-19T06:27:58.707 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1433450 13.40 MB/sec execute 462 sec latency 186.431 ms 2024-09-19T06:27:59.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:58 smithi053 ceph-mon[27968]: pgmap v554: 129 pgs: 129 active+clean; 181 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.7 MiB/s wr, 87 op/s 2024-09-19T06:27:59.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:58 smithi155 ceph-mon[35292]: pgmap v554: 129 pgs: 129 active+clean; 181 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.7 MiB/s wr, 87 op/s 2024-09-19T06:27:59.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 441716 4.06 MB/sec execute 466 sec latency 82.000 ms 2024-09-19T06:27:59.707 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1433706 13.38 MB/sec execute 463 sec latency 291.857 ms 2024-09-19T06:28:00.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:27:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:28:00.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:27:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:28:00.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 442457 4.05 MB/sec execute 467 sec latency 51.593 ms 2024-09-19T06:28:00.708 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1436161 13.38 MB/sec execute 464 sec latency 225.212 ms 2024-09-19T06:28:00.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:00 smithi053 ceph-mon[27968]: pgmap v555: 129 pgs: 129 active+clean; 182 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 2.1 MiB/s wr, 94 op/s 2024-09-19T06:28:01.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:00 smithi155 ceph-mon[35292]: pgmap v555: 129 pgs: 129 active+clean; 182 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 2.1 MiB/s wr, 94 op/s 2024-09-19T06:28:01.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 443247 4.06 MB/sec execute 468 sec latency 12.227 ms 2024-09-19T06:28:01.707 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1439639 13.39 MB/sec execute 465 sec latency 32.681 ms 2024-09-19T06:28:02.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 443950 4.06 MB/sec execute 469 sec latency 9.129 ms 2024-09-19T06:28:02.707 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1443136 13.40 MB/sec execute 466 sec latency 58.495 ms 2024-09-19T06:28:03.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:02 smithi053 ceph-mon[27968]: pgmap v556: 129 pgs: 129 active+clean; 183 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.6 MiB/s wr, 83 op/s 2024-09-19T06:28:03.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:02 smithi155 ceph-mon[35292]: pgmap v556: 129 pgs: 129 active+clean; 183 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.6 MiB/s wr, 83 op/s 2024-09-19T06:28:03.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 444787 4.06 MB/sec execute 470 sec latency 16.101 ms 2024-09-19T06:28:03.707 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1446671 13.41 MB/sec execute 467 sec latency 25.222 ms 2024-09-19T06:28:04.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 445628 4.06 MB/sec execute 471 sec latency 10.243 ms 2024-09-19T06:28:04.707 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1449222 13.41 MB/sec execute 468 sec latency 33.746 ms 2024-09-19T06:28:05.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:04 smithi053 ceph-mon[27968]: pgmap v557: 129 pgs: 129 active+clean; 182 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.8 MiB/s wr, 94 op/s 2024-09-19T06:28:05.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:04 smithi155 ceph-mon[35292]: pgmap v557: 129 pgs: 129 active+clean; 182 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.8 MiB/s wr, 94 op/s 2024-09-19T06:28:05.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 446614 4.06 MB/sec execute 472 sec latency 21.627 ms 2024-09-19T06:28:05.707 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1450376 13.40 MB/sec execute 469 sec latency 291.868 ms 2024-09-19T06:28:06.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 447246 4.07 MB/sec execute 473 sec latency 13.451 ms 2024-09-19T06:28:06.707 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1450383 13.37 MB/sec execute 470 sec latency 305.406 ms 2024-09-19T06:28:07.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:06 smithi053 ceph-mon[27968]: pgmap v558: 129 pgs: 129 active+clean; 184 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.9 MiB/s wr, 92 op/s 2024-09-19T06:28:07.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:06 smithi155 ceph-mon[35292]: pgmap v558: 129 pgs: 129 active+clean; 184 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.9 MiB/s wr, 92 op/s 2024-09-19T06:28:07.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 447811 4.06 MB/sec execute 474 sec latency 32.673 ms 2024-09-19T06:28:07.707 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1453800 13.38 MB/sec execute 471 sec latency 300.260 ms 2024-09-19T06:28:08.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 448596 4.06 MB/sec execute 475 sec latency 27.995 ms 2024-09-19T06:28:08.708 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1456444 13.38 MB/sec execute 472 sec latency 33.395 ms 2024-09-19T06:28:09.034 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:08 smithi155 ceph-mon[35292]: pgmap v559: 129 pgs: 129 active+clean; 183 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 89 op/s 2024-09-19T06:28:09.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:08 smithi053 ceph-mon[27968]: pgmap v559: 129 pgs: 129 active+clean; 183 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 89 op/s 2024-09-19T06:28:09.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 449431 4.06 MB/sec execute 476 sec latency 9.410 ms 2024-09-19T06:28:09.707 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1459646 13.39 MB/sec execute 473 sec latency 41.708 ms 2024-09-19T06:28:10.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 450271 4.06 MB/sec execute 477 sec latency 13.882 ms 2024-09-19T06:28:10.707 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1462586 13.39 MB/sec execute 474 sec latency 33.432 ms 2024-09-19T06:28:11.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:10 smithi053 ceph-mon[27968]: pgmap v560: 129 pgs: 129 active+clean; 186 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.2 MiB/s wr, 106 op/s 2024-09-19T06:28:11.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:10 smithi155 ceph-mon[35292]: pgmap v560: 129 pgs: 129 active+clean; 186 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.2 MiB/s wr, 106 op/s 2024-09-19T06:28:11.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 450917 4.06 MB/sec execute 478 sec latency 48.230 ms 2024-09-19T06:28:11.707 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1464570 13.39 MB/sec execute 475 sec latency 174.967 ms 2024-09-19T06:28:12.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 451563 4.06 MB/sec execute 479 sec latency 27.760 ms 2024-09-19T06:28:12.707 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1467083 13.39 MB/sec execute 476 sec latency 316.783 ms 2024-09-19T06:28:13.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:12 smithi053 ceph-mon[27968]: pgmap v561: 129 pgs: 129 active+clean; 185 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 79 op/s 2024-09-19T06:28:13.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:12 smithi155 ceph-mon[35292]: pgmap v561: 129 pgs: 129 active+clean; 185 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 79 op/s 2024-09-19T06:28:13.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 452399 4.06 MB/sec execute 480 sec latency 13.805 ms 2024-09-19T06:28:13.707 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1467980 13.37 MB/sec execute 477 sec latency 274.852 ms 2024-09-19T06:28:14.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 453254 4.06 MB/sec execute 481 sec latency 16.094 ms 2024-09-19T06:28:14.707 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1469781 13.36 MB/sec execute 478 sec latency 275.119 ms 2024-09-19T06:28:15.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:14 smithi053 ceph-mon[27968]: pgmap v562: 129 pgs: 129 active+clean; 186 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.0 MiB/s wr, 109 op/s 2024-09-19T06:28:15.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:14 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:28:15.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:14 smithi155 ceph-mon[35292]: pgmap v562: 129 pgs: 129 active+clean; 186 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.0 MiB/s wr, 109 op/s 2024-09-19T06:28:15.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:14 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:28:15.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 454068 4.07 MB/sec execute 482 sec latency 11.382 ms 2024-09-19T06:28:15.707 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1472738 13.37 MB/sec execute 479 sec latency 64.125 ms 2024-09-19T06:28:16.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 454650 4.06 MB/sec execute 483 sec latency 11.941 ms 2024-09-19T06:28:16.707 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1475206 13.37 MB/sec execute 480 sec latency 41.193 ms 2024-09-19T06:28:17.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:16 smithi053 ceph-mon[27968]: pgmap v563: 129 pgs: 129 active+clean; 187 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 2.0 MiB/s wr, 104 op/s 2024-09-19T06:28:17.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:16 smithi155 ceph-mon[35292]: pgmap v563: 129 pgs: 129 active+clean; 187 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 2.0 MiB/s wr, 104 op/s 2024-09-19T06:28:17.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 455449 4.06 MB/sec execute 484 sec latency 15.495 ms 2024-09-19T06:28:17.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1478686 13.38 MB/sec execute 481 sec latency 53.501 ms 2024-09-19T06:28:18.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 456333 4.06 MB/sec execute 485 sec latency 12.237 ms 2024-09-19T06:28:18.707 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1482217 13.39 MB/sec execute 482 sec latency 33.725 ms 2024-09-19T06:28:19.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:18 smithi053 ceph-mon[27968]: pgmap v564: 129 pgs: 129 active+clean; 187 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.7 MiB/s wr, 81 op/s 2024-09-19T06:28:19.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:18 smithi155 ceph-mon[35292]: pgmap v564: 129 pgs: 129 active+clean; 187 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.7 MiB/s wr, 81 op/s 2024-09-19T06:28:19.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 457060 4.06 MB/sec execute 486 sec latency 89.313 ms 2024-09-19T06:28:19.707 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1482295 13.37 MB/sec execute 483 sec latency 291.872 ms 2024-09-19T06:28:20.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:19 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:28:20.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:19 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:28:20.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:19 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:28:20.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:19 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:28:20.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:19 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:28:20.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:19 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:28:20.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:19 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:28:20.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:19 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:28:20.632 INFO:tasks.workunit.client.1.smithi155.stdout: 1 457784 4.07 MB/sec execute 487 sec latency 39.920 ms 2024-09-19T06:28:20.707 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1484250 13.36 MB/sec execute 484 sec latency 308.570 ms 2024-09-19T06:28:21.101 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:20 smithi053 ceph-mon[27968]: pgmap v565: 129 pgs: 129 active+clean; 188 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.2 MiB/s wr, 105 op/s 2024-09-19T06:28:21.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:20 smithi155 ceph-mon[35292]: pgmap v565: 129 pgs: 129 active+clean; 188 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.2 MiB/s wr, 105 op/s 2024-09-19T06:28:21.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 458312 4.06 MB/sec execute 488 sec latency 15.633 ms 2024-09-19T06:28:21.708 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1486924 13.36 MB/sec execute 485 sec latency 101.653 ms 2024-09-19T06:28:22.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 459167 4.09 MB/sec execute 489 sec latency 44.672 ms 2024-09-19T06:28:22.708 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1489565 13.36 MB/sec execute 486 sec latency 94.822 ms 2024-09-19T06:28:23.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:22 smithi053 ceph-mon[27968]: pgmap v566: 129 pgs: 129 active+clean; 188 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.5 MiB/s wr, 79 op/s 2024-09-19T06:28:23.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:22 smithi155 ceph-mon[35292]: pgmap v566: 129 pgs: 129 active+clean; 188 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.5 MiB/s wr, 79 op/s 2024-09-19T06:28:23.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 459954 4.10 MB/sec execute 490 sec latency 22.455 ms 2024-09-19T06:28:23.708 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1493005 13.37 MB/sec execute 487 sec latency 49.776 ms 2024-09-19T06:28:24.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 460783 4.09 MB/sec execute 491 sec latency 12.367 ms 2024-09-19T06:28:24.708 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1496405 13.39 MB/sec execute 488 sec latency 135.494 ms 2024-09-19T06:28:25.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:24 smithi053 ceph-mon[27968]: pgmap v567: 129 pgs: 129 active+clean; 198 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.7 MiB/s wr, 113 op/s 2024-09-19T06:28:25.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:24 smithi155 ceph-mon[35292]: pgmap v567: 129 pgs: 129 active+clean; 198 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.7 MiB/s wr, 113 op/s 2024-09-19T06:28:25.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 461667 4.10 MB/sec execute 492 sec latency 11.705 ms 2024-09-19T06:28:25.708 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1499947 13.40 MB/sec execute 489 sec latency 33.410 ms 2024-09-19T06:28:26.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 462322 4.10 MB/sec execute 493 sec latency 11.574 ms 2024-09-19T06:28:26.708 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1503437 13.41 MB/sec execute 490 sec latency 49.683 ms 2024-09-19T06:28:27.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:26 smithi053 ceph-mon[27968]: pgmap v568: 129 pgs: 129 active+clean; 198 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 3.1 MiB/s wr, 90 op/s 2024-09-19T06:28:27.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:26 smithi155 ceph-mon[35292]: pgmap v568: 129 pgs: 129 active+clean; 198 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 3.1 MiB/s wr, 90 op/s 2024-09-19T06:28:27.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 463007 4.09 MB/sec execute 494 sec latency 15.521 ms 2024-09-19T06:28:27.708 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1506055 13.41 MB/sec execute 491 sec latency 41.825 ms 2024-09-19T06:28:28.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 463829 4.09 MB/sec execute 495 sec latency 11.977 ms 2024-09-19T06:28:28.708 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1508788 13.41 MB/sec execute 492 sec latency 35.255 ms 2024-09-19T06:28:29.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:28 smithi053 ceph-mon[27968]: pgmap v569: 129 pgs: 129 active+clean; 198 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 3.2 MiB/s wr, 90 op/s 2024-09-19T06:28:29.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:28 smithi155 ceph-mon[35292]: pgmap v569: 129 pgs: 129 active+clean; 198 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 3.2 MiB/s wr, 90 op/s 2024-09-19T06:28:29.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 464774 4.10 MB/sec execute 496 sec latency 22.837 ms 2024-09-19T06:28:29.708 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1511742 13.41 MB/sec execute 493 sec latency 69.357 ms 2024-09-19T06:28:30.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:28:30.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:29 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:28:30.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 465534 4.10 MB/sec execute 497 sec latency 17.271 ms 2024-09-19T06:28:30.708 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1514389 13.42 MB/sec execute 494 sec latency 58.676 ms 2024-09-19T06:28:31.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:30 smithi053 ceph-mon[27968]: pgmap v570: 129 pgs: 129 active+clean; 191 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 5.4 KiB/s rd, 3.7 MiB/s wr, 115 op/s 2024-09-19T06:28:31.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:30 smithi155 ceph-mon[35292]: pgmap v570: 129 pgs: 129 active+clean; 191 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 5.4 KiB/s rd, 3.7 MiB/s wr, 115 op/s 2024-09-19T06:28:31.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 466103 4.10 MB/sec execute 498 sec latency 18.277 ms 2024-09-19T06:28:31.708 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1517743 13.43 MB/sec execute 495 sec latency 111.672 ms 2024-09-19T06:28:32.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 466938 4.10 MB/sec execute 499 sec latency 12.674 ms 2024-09-19T06:28:32.708 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1521229 13.44 MB/sec execute 496 sec latency 33.450 ms 2024-09-19T06:28:33.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:32 smithi053 ceph-mon[27968]: pgmap v571: 129 pgs: 129 active+clean; 191 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.2 MiB/s wr, 88 op/s 2024-09-19T06:28:33.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:32 smithi155 ceph-mon[35292]: pgmap v571: 129 pgs: 129 active+clean; 191 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.2 MiB/s wr, 88 op/s 2024-09-19T06:28:33.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 467812 4.09 MB/sec execute 500 sec latency 9.689 ms 2024-09-19T06:28:33.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1524716 13.45 MB/sec execute 497 sec latency 35.267 ms 2024-09-19T06:28:34.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 468749 4.10 MB/sec execute 501 sec latency 12.196 ms 2024-09-19T06:28:34.708 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1527031 13.45 MB/sec execute 498 sec latency 58.280 ms 2024-09-19T06:28:35.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:34 smithi053 ceph-mon[27968]: pgmap v572: 129 pgs: 129 active+clean; 191 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 5.4 KiB/s rd, 3.7 MiB/s wr, 117 op/s 2024-09-19T06:28:35.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:34 smithi155 ceph-mon[35292]: pgmap v572: 129 pgs: 129 active+clean; 191 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 5.4 KiB/s rd, 3.7 MiB/s wr, 117 op/s 2024-09-19T06:28:35.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 469373 4.10 MB/sec execute 502 sec latency 13.723 ms 2024-09-19T06:28:35.708 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1529928 13.45 MB/sec execute 499 sec latency 43.475 ms 2024-09-19T06:28:36.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 470048 4.10 MB/sec execute 503 sec latency 15.061 ms 2024-09-19T06:28:36.708 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1533015 13.46 MB/sec execute 500 sec latency 33.185 ms 2024-09-19T06:28:37.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:36 smithi053 ceph-mon[27968]: pgmap v573: 129 pgs: 129 active+clean; 192 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 3.7 KiB/s rd, 2.8 MiB/s wr, 89 op/s 2024-09-19T06:28:37.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:36 smithi155 ceph-mon[35292]: pgmap v573: 129 pgs: 129 active+clean; 192 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 3.7 KiB/s rd, 2.8 MiB/s wr, 89 op/s 2024-09-19T06:28:37.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 470905 4.10 MB/sec execute 504 sec latency 12.062 ms 2024-09-19T06:28:37.708 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1535598 13.46 MB/sec execute 501 sec latency 58.733 ms 2024-09-19T06:28:38.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 471771 4.10 MB/sec execute 505 sec latency 20.414 ms 2024-09-19T06:28:38.708 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1539102 13.47 MB/sec execute 502 sec latency 33.084 ms 2024-09-19T06:28:39.034 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:38 smithi155 ceph-mon[35292]: pgmap v574: 129 pgs: 129 active+clean; 193 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 3.7 KiB/s rd, 2.0 MiB/s wr, 86 op/s 2024-09-19T06:28:39.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:38 smithi053 ceph-mon[27968]: pgmap v574: 129 pgs: 129 active+clean; 193 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 3.7 KiB/s rd, 2.0 MiB/s wr, 86 op/s 2024-09-19T06:28:39.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 472636 4.10 MB/sec execute 506 sec latency 15.046 ms 2024-09-19T06:28:39.708 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1542574 13.48 MB/sec execute 503 sec latency 33.568 ms 2024-09-19T06:28:40.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 473208 4.10 MB/sec execute 507 sec latency 14.714 ms 2024-09-19T06:28:40.708 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1546105 13.49 MB/sec execute 504 sec latency 33.343 ms 2024-09-19T06:28:41.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:40 smithi053 ceph-mon[27968]: pgmap v575: 129 pgs: 129 active+clean; 194 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 5.4 KiB/s rd, 2.2 MiB/s wr, 109 op/s 2024-09-19T06:28:41.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:40 smithi155 ceph-mon[35292]: pgmap v575: 129 pgs: 129 active+clean; 194 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 5.4 KiB/s rd, 2.2 MiB/s wr, 109 op/s 2024-09-19T06:28:41.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 474031 4.10 MB/sec execute 508 sec latency 24.267 ms 2024-09-19T06:28:41.708 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1548304 13.49 MB/sec execute 505 sec latency 199.997 ms 2024-09-19T06:28:42.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 474901 4.10 MB/sec execute 509 sec latency 14.838 ms 2024-09-19T06:28:42.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1550748 13.49 MB/sec execute 506 sec latency 100.108 ms 2024-09-19T06:28:43.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:42 smithi053 ceph-mon[27968]: pgmap v576: 129 pgs: 129 active+clean; 193 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.6 MiB/s wr, 82 op/s 2024-09-19T06:28:43.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:42 smithi155 ceph-mon[35292]: pgmap v576: 129 pgs: 129 active+clean; 193 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.6 MiB/s wr, 82 op/s 2024-09-19T06:28:43.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 475842 4.10 MB/sec execute 510 sec latency 16.307 ms 2024-09-19T06:28:43.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1554036 13.50 MB/sec execute 507 sec latency 33.389 ms 2024-09-19T06:28:44.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:43 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:28:44.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:43 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:28:44.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 476455 4.10 MB/sec execute 511 sec latency 13.463 ms 2024-09-19T06:28:44.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1556934 13.50 MB/sec execute 508 sec latency 55.807 ms 2024-09-19T06:28:45.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:44 smithi053 ceph-mon[27968]: pgmap v577: 129 pgs: 129 active+clean; 195 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.2 MiB/s wr, 110 op/s 2024-09-19T06:28:45.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:44 smithi155 ceph-mon[35292]: pgmap v577: 129 pgs: 129 active+clean; 195 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.2 MiB/s wr, 110 op/s 2024-09-19T06:28:45.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 477130 4.10 MB/sec execute 512 sec latency 15.142 ms 2024-09-19T06:28:45.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1560371 13.51 MB/sec execute 509 sec latency 33.726 ms 2024-09-19T06:28:46.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 477972 4.10 MB/sec execute 513 sec latency 11.794 ms 2024-09-19T06:28:46.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1563777 13.52 MB/sec execute 510 sec latency 175.019 ms 2024-09-19T06:28:47.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:46 smithi053 ceph-mon[27968]: pgmap v578: 129 pgs: 129 active+clean; 195 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.9 MiB/s wr, 89 op/s 2024-09-19T06:28:47.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:46 smithi155 ceph-mon[35292]: pgmap v578: 129 pgs: 129 active+clean; 195 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.9 MiB/s wr, 89 op/s 2024-09-19T06:28:47.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 478721 4.10 MB/sec execute 514 sec latency 36.205 ms 2024-09-19T06:28:47.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1563840 13.49 MB/sec execute 511 sec latency 300.261 ms 2024-09-19T06:28:48.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 479513 4.10 MB/sec execute 515 sec latency 38.883 ms 2024-09-19T06:28:48.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1563850 13.47 MB/sec execute 512 sec latency 283.539 ms 2024-09-19T06:28:49.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:48 smithi053 ceph-mon[27968]: pgmap v579: 129 pgs: 129 active+clean; 195 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.7 MiB/s wr, 85 op/s 2024-09-19T06:28:49.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:48 smithi155 ceph-mon[35292]: pgmap v579: 129 pgs: 129 active+clean; 195 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.7 MiB/s wr, 85 op/s 2024-09-19T06:28:49.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 480147 4.10 MB/sec execute 516 sec latency 36.746 ms 2024-09-19T06:28:49.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1563961 13.44 MB/sec execute 513 sec latency 308.533 ms 2024-09-19T06:28:50.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 480842 4.10 MB/sec execute 517 sec latency 31.309 ms 2024-09-19T06:28:50.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1564526 13.42 MB/sec execute 514 sec latency 291.790 ms 2024-09-19T06:28:51.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:50 smithi053 ceph-mon[27968]: pgmap v580: 129 pgs: 129 active+clean; 197 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 5.3 KiB/s rd, 2.3 MiB/s wr, 109 op/s 2024-09-19T06:28:51.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:50 smithi155 ceph-mon[35292]: pgmap v580: 129 pgs: 129 active+clean; 197 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 5.3 KiB/s rd, 2.3 MiB/s wr, 109 op/s 2024-09-19T06:28:51.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 481543 4.10 MB/sec execute 518 sec latency 69.384 ms 2024-09-19T06:28:51.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1564928 13.40 MB/sec execute 515 sec latency 329.223 ms 2024-09-19T06:28:52.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 482228 4.09 MB/sec execute 519 sec latency 102.681 ms 2024-09-19T06:28:52.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1565416 13.38 MB/sec execute 516 sec latency 316.901 ms 2024-09-19T06:28:53.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:52 smithi053 ceph-mon[27968]: pgmap v581: 129 pgs: 129 active+clean; 197 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.7 MiB/s wr, 83 op/s 2024-09-19T06:28:53.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:52 smithi155 ceph-mon[35292]: pgmap v581: 129 pgs: 129 active+clean; 197 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.7 MiB/s wr, 83 op/s 2024-09-19T06:28:53.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 483041 4.10 MB/sec execute 520 sec latency 14.007 ms 2024-09-19T06:28:53.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1565804 13.36 MB/sec execute 517 sec latency 283.559 ms 2024-09-19T06:28:54.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 483681 4.10 MB/sec execute 521 sec latency 67.532 ms 2024-09-19T06:28:54.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1567384 13.35 MB/sec execute 518 sec latency 331.996 ms 2024-09-19T06:28:55.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:54 smithi053 ceph-mon[27968]: pgmap v582: 129 pgs: 129 active+clean; 197 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 5.7 KiB/s rd, 2.1 MiB/s wr, 110 op/s 2024-09-19T06:28:55.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:54 smithi155 ceph-mon[35292]: pgmap v582: 129 pgs: 129 active+clean; 197 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 5.7 KiB/s rd, 2.1 MiB/s wr, 110 op/s 2024-09-19T06:28:55.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 484395 4.10 MB/sec execute 522 sec latency 16.388 ms 2024-09-19T06:28:55.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1570812 13.36 MB/sec execute 519 sec latency 150.102 ms 2024-09-19T06:28:56.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 485303 4.09 MB/sec execute 523 sec latency 12.191 ms 2024-09-19T06:28:56.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1573462 13.36 MB/sec execute 520 sec latency 33.512 ms 2024-09-19T06:28:57.051 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:56 smithi053 ceph-mon[27968]: pgmap v583: 129 pgs: 129 active+clean; 197 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 3.8 KiB/s rd, 1.7 MiB/s wr, 89 op/s 2024-09-19T06:28:57.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:56 smithi155 ceph-mon[35292]: pgmap v583: 129 pgs: 129 active+clean; 197 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 3.8 KiB/s rd, 1.7 MiB/s wr, 89 op/s 2024-09-19T06:28:57.633 INFO:tasks.workunit.client.1.smithi155.stdout: 1 486193 4.10 MB/sec execute 524 sec latency 18.954 ms 2024-09-19T06:28:57.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1576668 13.37 MB/sec execute 521 sec latency 34.819 ms 2024-09-19T06:28:58.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 486974 4.10 MB/sec execute 525 sec latency 20.720 ms 2024-09-19T06:28:58.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1579637 13.37 MB/sec execute 522 sec latency 40.994 ms 2024-09-19T06:28:59.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:58 smithi053 ceph-mon[27968]: pgmap v584: 129 pgs: 129 active+clean; 199 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 3.8 KiB/s rd, 1.7 MiB/s wr, 100 op/s 2024-09-19T06:28:59.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:28:58 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:28:59.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:58 smithi155 ceph-mon[35292]: pgmap v584: 129 pgs: 129 active+clean; 199 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 3.8 KiB/s rd, 1.7 MiB/s wr, 100 op/s 2024-09-19T06:28:59.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:28:58 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:28:59.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 487484 4.10 MB/sec execute 526 sec latency 18.054 ms 2024-09-19T06:28:59.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1581569 13.37 MB/sec execute 523 sec latency 283.731 ms 2024-09-19T06:29:00.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 488164 4.10 MB/sec execute 527 sec latency 193.136 ms 2024-09-19T06:29:00.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1581574 13.34 MB/sec execute 524 sec latency 593.758 ms 2024-09-19T06:29:01.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:00 smithi053 ceph-mon[27968]: pgmap v585: 129 pgs: 129 active+clean; 200 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.1 MiB/s wr, 107 op/s 2024-09-19T06:29:01.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:00 smithi155 ceph-mon[35292]: pgmap v585: 129 pgs: 129 active+clean; 200 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.1 MiB/s wr, 107 op/s 2024-09-19T06:29:01.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 489055 4.09 MB/sec execute 528 sec latency 17.522 ms 2024-09-19T06:29:01.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1583107 13.33 MB/sec execute 525 sec latency 281.973 ms 2024-09-19T06:29:02.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 490039 4.10 MB/sec execute 529 sec latency 23.081 ms 2024-09-19T06:29:02.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1585950 13.34 MB/sec execute 526 sec latency 51.794 ms 2024-09-19T06:29:03.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:02 smithi053 ceph-mon[27968]: pgmap v586: 129 pgs: 129 active+clean; 199 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.5 MiB/s wr, 80 op/s 2024-09-19T06:29:03.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:02 smithi155 ceph-mon[35292]: pgmap v586: 129 pgs: 129 active+clean; 199 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.5 MiB/s wr, 80 op/s 2024-09-19T06:29:03.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 490586 4.10 MB/sec execute 530 sec latency 14.066 ms 2024-09-19T06:29:03.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1588843 13.34 MB/sec execute 527 sec latency 41.766 ms 2024-09-19T06:29:04.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 491133 4.10 MB/sec execute 531 sec latency 48.046 ms 2024-09-19T06:29:04.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1592133 13.35 MB/sec execute 528 sec latency 300.525 ms 2024-09-19T06:29:05.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:04 smithi053 ceph-mon[27968]: pgmap v587: 129 pgs: 129 active+clean; 200 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.3 KiB/s rd, 2.1 MiB/s wr, 108 op/s 2024-09-19T06:29:05.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:04 smithi155 ceph-mon[35292]: pgmap v587: 129 pgs: 129 active+clean; 200 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.3 KiB/s rd, 2.1 MiB/s wr, 108 op/s 2024-09-19T06:29:05.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 491946 4.10 MB/sec execute 532 sec latency 14.718 ms 2024-09-19T06:29:05.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1594748 13.35 MB/sec execute 529 sec latency 55.306 ms 2024-09-19T06:29:06.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 492767 4.09 MB/sec execute 533 sec latency 11.728 ms 2024-09-19T06:29:06.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1597344 13.35 MB/sec execute 530 sec latency 33.564 ms 2024-09-19T06:29:07.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:06 smithi053 ceph-mon[27968]: pgmap v588: 129 pgs: 129 active+clean; 200 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 MiB/s wr, 84 op/s 2024-09-19T06:29:07.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:06 smithi155 ceph-mon[35292]: pgmap v588: 129 pgs: 129 active+clean; 200 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 MiB/s wr, 84 op/s 2024-09-19T06:29:07.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 493640 4.10 MB/sec execute 534 sec latency 12.910 ms 2024-09-19T06:29:07.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1600435 13.36 MB/sec execute 531 sec latency 41.647 ms 2024-09-19T06:29:08.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 494289 4.10 MB/sec execute 535 sec latency 12.099 ms 2024-09-19T06:29:08.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1603576 13.37 MB/sec execute 532 sec latency 35.064 ms 2024-09-19T06:29:09.034 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:08 smithi155 ceph-mon[35292]: pgmap v589: 129 pgs: 129 active+clean; 201 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 MiB/s wr, 92 op/s 2024-09-19T06:29:09.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:08 smithi053 ceph-mon[27968]: pgmap v589: 129 pgs: 129 active+clean; 201 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 MiB/s wr, 92 op/s 2024-09-19T06:29:09.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 494984 4.10 MB/sec execute 536 sec latency 16.417 ms 2024-09-19T06:29:09.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1606437 13.37 MB/sec execute 533 sec latency 125.155 ms 2024-09-19T06:29:10.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 495726 4.09 MB/sec execute 537 sec latency 79.555 ms 2024-09-19T06:29:10.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1606446 13.34 MB/sec execute 534 sec latency 275.106 ms 2024-09-19T06:29:11.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:10 smithi053 ceph-mon[27968]: pgmap v590: 129 pgs: 129 active+clean; 202 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.1 MiB/s wr, 92 op/s 2024-09-19T06:29:11.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:10 smithi155 ceph-mon[35292]: pgmap v590: 129 pgs: 129 active+clean; 202 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.1 MiB/s wr, 92 op/s 2024-09-19T06:29:11.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 496411 4.09 MB/sec execute 538 sec latency 93.469 ms 2024-09-19T06:29:11.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1606523 13.32 MB/sec execute 535 sec latency 425.396 ms 2024-09-19T06:29:12.637 INFO:tasks.workunit.client.1.smithi155.stdout: 1 497191 4.10 MB/sec execute 539 sec latency 52.610 ms 2024-09-19T06:29:12.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1609865 13.33 MB/sec execute 536 sec latency 258.577 ms 2024-09-19T06:29:13.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:12 smithi053 ceph-mon[27968]: pgmap v591: 129 pgs: 129 active+clean; 202 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.7 MiB/s wr, 83 op/s 2024-09-19T06:29:13.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:12 smithi155 ceph-mon[35292]: pgmap v591: 129 pgs: 129 active+clean; 202 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.7 MiB/s wr, 83 op/s 2024-09-19T06:29:13.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 497886 4.10 MB/sec execute 540 sec latency 12.441 ms 2024-09-19T06:29:13.709 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1612875 13.34 MB/sec execute 537 sec latency 33.557 ms 2024-09-19T06:29:14.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:13 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:29:14.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:13 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:29:14.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 498620 4.09 MB/sec execute 541 sec latency 21.125 ms 2024-09-19T06:29:14.710 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1615922 13.34 MB/sec execute 538 sec latency 35.189 ms 2024-09-19T06:29:15.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:14 smithi053 ceph-mon[27968]: pgmap v592: 129 pgs: 129 active+clean; 202 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.1 MiB/s wr, 111 op/s 2024-09-19T06:29:15.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:14 smithi155 ceph-mon[35292]: pgmap v592: 129 pgs: 129 active+clean; 202 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.1 MiB/s wr, 111 op/s 2024-09-19T06:29:15.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 499455 4.09 MB/sec execute 542 sec latency 39.665 ms 2024-09-19T06:29:15.710 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1617073 13.33 MB/sec execute 539 sec latency 366.863 ms 2024-09-19T06:29:16.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 500320 4.10 MB/sec execute 543 sec latency 13.086 ms 2024-09-19T06:29:16.710 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1617083 13.30 MB/sec execute 540 sec latency 341.863 ms 2024-09-19T06:29:17.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:16 smithi053 ceph-mon[27968]: pgmap v593: 129 pgs: 129 active+clean; 202 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.6 MiB/s wr, 84 op/s 2024-09-19T06:29:17.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:16 smithi155 ceph-mon[35292]: pgmap v593: 129 pgs: 129 active+clean; 202 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.6 MiB/s wr, 84 op/s 2024-09-19T06:29:17.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 501065 4.10 MB/sec execute 544 sec latency 13.260 ms 2024-09-19T06:29:17.710 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1620619 13.31 MB/sec execute 541 sec latency 150.222 ms 2024-09-19T06:29:18.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 501603 4.09 MB/sec execute 545 sec latency 43.930 ms 2024-09-19T06:29:18.710 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1624103 13.32 MB/sec execute 542 sec latency 33.713 ms 2024-09-19T06:29:19.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:18 smithi053 ceph-mon[27968]: pgmap v594: 129 pgs: 129 active+clean; 202 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.6 MiB/s wr, 85 op/s 2024-09-19T06:29:19.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:18 smithi155 ceph-mon[35292]: pgmap v594: 129 pgs: 129 active+clean; 202 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.6 MiB/s wr, 85 op/s 2024-09-19T06:29:19.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 502433 4.10 MB/sec execute 546 sec latency 11.920 ms 2024-09-19T06:29:19.710 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1625707 13.31 MB/sec execute 543 sec latency 250.911 ms 2024-09-19T06:29:20.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:19 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:29:20.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:19 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:29:20.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 503278 4.09 MB/sec execute 547 sec latency 17.263 ms 2024-09-19T06:29:20.710 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1627838 13.31 MB/sec execute 544 sec latency 51.300 ms 2024-09-19T06:29:21.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:20 smithi053 ceph-mon[27968]: pgmap v595: 129 pgs: 129 active+clean; 204 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.2 MiB/s wr, 94 op/s 2024-09-19T06:29:21.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:20 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:29:21.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:20 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:29:21.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:20 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:29:21.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:20 smithi155 ceph-mon[35292]: pgmap v595: 129 pgs: 129 active+clean; 204 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.2 MiB/s wr, 94 op/s 2024-09-19T06:29:21.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:20 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:29:21.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:20 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:29:21.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:20 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:29:21.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 504231 4.10 MB/sec execute 548 sec latency 13.839 ms 2024-09-19T06:29:21.710 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1631262 13.32 MB/sec execute 545 sec latency 41.460 ms 2024-09-19T06:29:22.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 504803 4.10 MB/sec execute 549 sec latency 44.089 ms 2024-09-19T06:29:22.710 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1632831 13.31 MB/sec execute 546 sec latency 292.322 ms 2024-09-19T06:29:23.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:22 smithi053 ceph-mon[27968]: pgmap v596: 129 pgs: 129 active+clean; 204 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.5 MiB/s wr, 78 op/s 2024-09-19T06:29:23.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:22 smithi155 ceph-mon[35292]: pgmap v596: 129 pgs: 129 active+clean; 204 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.5 MiB/s wr, 78 op/s 2024-09-19T06:29:23.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 505363 4.09 MB/sec execute 550 sec latency 43.373 ms 2024-09-19T06:29:23.710 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1633373 13.29 MB/sec execute 547 sec latency 266.189 ms 2024-09-19T06:29:24.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 506183 4.09 MB/sec execute 551 sec latency 13.389 ms 2024-09-19T06:29:24.710 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1634994 13.28 MB/sec execute 548 sec latency 187.352 ms 2024-09-19T06:29:25.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:24 smithi053 ceph-mon[27968]: pgmap v597: 129 pgs: 129 active+clean; 205 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 2.1 MiB/s wr, 107 op/s 2024-09-19T06:29:25.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:24 smithi155 ceph-mon[35292]: pgmap v597: 129 pgs: 129 active+clean; 205 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 2.1 MiB/s wr, 107 op/s 2024-09-19T06:29:25.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 506981 4.09 MB/sec execute 552 sec latency 12.323 ms 2024-09-19T06:29:25.710 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1638431 13.29 MB/sec execute 549 sec latency 33.338 ms 2024-09-19T06:29:26.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 507817 4.10 MB/sec execute 553 sec latency 23.722 ms 2024-09-19T06:29:26.710 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1641898 13.30 MB/sec execute 550 sec latency 33.782 ms 2024-09-19T06:29:27.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:26 smithi053 ceph-mon[27968]: pgmap v598: 129 pgs: 129 active+clean; 205 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.8 MiB/s wr, 81 op/s 2024-09-19T06:29:27.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:26 smithi155 ceph-mon[35292]: pgmap v598: 129 pgs: 129 active+clean; 205 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.8 MiB/s wr, 81 op/s 2024-09-19T06:29:27.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 508493 4.10 MB/sec execute 554 sec latency 12.622 ms 2024-09-19T06:29:27.710 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1645323 13.31 MB/sec execute 551 sec latency 63.313 ms 2024-09-19T06:29:28.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:27 smithi053 ceph-mon[27968]: pgmap v599: 129 pgs: 129 active+clean; 208 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.9 MiB/s wr, 97 op/s 2024-09-19T06:29:28.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:27 smithi155 ceph-mon[35292]: pgmap v599: 129 pgs: 129 active+clean; 208 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.9 MiB/s wr, 97 op/s 2024-09-19T06:29:28.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 509203 4.09 MB/sec execute 555 sec latency 15.734 ms 2024-09-19T06:29:28.710 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1648860 13.32 MB/sec execute 552 sec latency 33.469 ms 2024-09-19T06:29:29.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:28 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:29:29.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:28 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:29:29.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 510074 4.09 MB/sec execute 556 sec latency 14.746 ms 2024-09-19T06:29:29.710 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1651162 13.32 MB/sec execute 553 sec latency 33.523 ms 2024-09-19T06:29:30.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:29 smithi155 ceph-mon[35292]: pgmap v600: 129 pgs: 129 active+clean; 207 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.2 MiB/s wr, 103 op/s 2024-09-19T06:29:30.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:29 smithi053 ceph-mon[27968]: pgmap v600: 129 pgs: 129 active+clean; 207 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.2 MiB/s wr, 103 op/s 2024-09-19T06:29:30.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 510800 4.09 MB/sec execute 557 sec latency 120.993 ms 2024-09-19T06:29:30.713 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1652458 13.31 MB/sec execute 554 sec latency 451.224 ms 2024-09-19T06:29:31.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 511616 4.10 MB/sec execute 558 sec latency 18.755 ms 2024-09-19T06:29:31.710 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1655956 13.32 MB/sec execute 555 sec latency 34.641 ms 2024-09-19T06:29:32.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:32 smithi053 ceph-mon[27968]: pgmap v601: 129 pgs: 129 active+clean; 207 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.5 MiB/s wr, 77 op/s 2024-09-19T06:29:32.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:32 smithi155 ceph-mon[35292]: pgmap v601: 129 pgs: 129 active+clean; 207 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.5 MiB/s wr, 77 op/s 2024-09-19T06:29:32.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 512207 4.09 MB/sec execute 559 sec latency 20.699 ms 2024-09-19T06:29:32.710 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1658599 13.32 MB/sec execute 556 sec latency 33.808 ms 2024-09-19T06:29:33.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 513033 4.10 MB/sec execute 560 sec latency 11.950 ms 2024-09-19T06:29:33.710 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1659629 13.31 MB/sec execute 557 sec latency 328.138 ms 2024-09-19T06:29:34.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:34 smithi053 ceph-mon[27968]: pgmap v602: 129 pgs: 129 active+clean; 209 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 2.2 MiB/s wr, 108 op/s 2024-09-19T06:29:34.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:34 smithi155 ceph-mon[35292]: pgmap v602: 129 pgs: 129 active+clean; 209 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 2.2 MiB/s wr, 108 op/s 2024-09-19T06:29:34.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 513916 4.09 MB/sec execute 561 sec latency 11.591 ms 2024-09-19T06:29:34.710 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1663165 13.32 MB/sec execute 558 sec latency 49.787 ms 2024-09-19T06:29:35.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 514870 4.10 MB/sec execute 562 sec latency 20.928 ms 2024-09-19T06:29:35.710 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1666645 13.33 MB/sec execute 559 sec latency 33.441 ms 2024-09-19T06:29:36.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:36 smithi053 ceph-mon[27968]: pgmap v603: 129 pgs: 129 active+clean; 208 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.7 MiB/s wr, 81 op/s 2024-09-19T06:29:36.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:36 smithi155 ceph-mon[35292]: pgmap v603: 129 pgs: 129 active+clean; 208 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.7 MiB/s wr, 81 op/s 2024-09-19T06:29:36.634 INFO:tasks.workunit.client.1.smithi155.stdout: 1 515408 4.10 MB/sec execute 563 sec latency 77.140 ms 2024-09-19T06:29:36.710 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1666715 13.30 MB/sec execute 560 sec latency 349.998 ms 2024-09-19T06:29:37.635 INFO:tasks.workunit.client.1.smithi155.stdout: 1 515947 4.09 MB/sec execute 564 sec latency 53.578 ms 2024-09-19T06:29:37.710 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1666794 13.28 MB/sec execute 561 sec latency 274.713 ms 2024-09-19T06:29:38.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:38 smithi053 ceph-mon[27968]: pgmap v604: 129 pgs: 129 active+clean; 210 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.7 MiB/s wr, 97 op/s 2024-09-19T06:29:38.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:38 smithi155 ceph-mon[35292]: pgmap v604: 129 pgs: 129 active+clean; 210 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.7 MiB/s wr, 97 op/s 2024-09-19T06:29:38.635 INFO:tasks.workunit.client.1.smithi155.stdout: 1 516657 4.09 MB/sec execute 565 sec latency 48.868 ms 2024-09-19T06:29:38.710 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1666900 13.26 MB/sec execute 562 sec latency 258.494 ms 2024-09-19T06:29:39.635 INFO:tasks.workunit.client.1.smithi155.stdout: 1 517478 4.09 MB/sec execute 566 sec latency 42.478 ms 2024-09-19T06:29:39.710 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1670256 13.27 MB/sec execute 563 sec latency 300.113 ms 2024-09-19T06:29:40.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:40 smithi053 ceph-mon[27968]: pgmap v605: 129 pgs: 129 active+clean; 210 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.0 MiB/s wr, 93 op/s 2024-09-19T06:29:40.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:40 smithi155 ceph-mon[35292]: pgmap v605: 129 pgs: 129 active+clean; 210 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.0 MiB/s wr, 93 op/s 2024-09-19T06:29:40.635 INFO:tasks.workunit.client.1.smithi155.stdout: 1 518413 4.10 MB/sec execute 567 sec latency 26.187 ms 2024-09-19T06:29:40.711 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1671842 13.25 MB/sec execute 564 sec latency 308.763 ms 2024-09-19T06:29:41.635 INFO:tasks.workunit.client.1.smithi155.stdout: 1 518997 4.10 MB/sec execute 568 sec latency 12.068 ms 2024-09-19T06:29:41.711 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1674643 13.26 MB/sec execute 565 sec latency 50.038 ms 2024-09-19T06:29:42.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:42 smithi053 ceph-mon[27968]: pgmap v606: 129 pgs: 129 active+clean; 210 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.6 MiB/s wr, 85 op/s 2024-09-19T06:29:42.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:42 smithi155 ceph-mon[35292]: pgmap v606: 129 pgs: 129 active+clean; 210 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.6 MiB/s wr, 85 op/s 2024-09-19T06:29:42.635 INFO:tasks.workunit.client.1.smithi155.stdout: 1 519611 4.09 MB/sec execute 569 sec latency 16.114 ms 2024-09-19T06:29:42.711 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1677478 13.26 MB/sec execute 566 sec latency 33.342 ms 2024-09-19T06:29:43.635 INFO:tasks.workunit.client.1.smithi155.stdout: 1 520441 4.09 MB/sec execute 570 sec latency 11.476 ms 2024-09-19T06:29:43.711 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1680907 13.27 MB/sec execute 567 sec latency 33.472 ms 2024-09-19T06:29:44.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:44 smithi053 ceph-mon[27968]: pgmap v607: 129 pgs: 129 active+clean; 212 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.2 MiB/s wr, 116 op/s 2024-09-19T06:29:44.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:44 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:29:44.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:44 smithi155 ceph-mon[35292]: pgmap v607: 129 pgs: 129 active+clean; 212 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.2 MiB/s wr, 116 op/s 2024-09-19T06:29:44.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:44 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:29:44.635 INFO:tasks.workunit.client.1.smithi155.stdout: 1 521225 4.09 MB/sec execute 571 sec latency 19.145 ms 2024-09-19T06:29:44.711 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1684386 13.28 MB/sec execute 568 sec latency 49.908 ms 2024-09-19T06:29:45.635 INFO:tasks.workunit.client.1.smithi155.stdout: 1 522030 4.09 MB/sec execute 572 sec latency 14.782 ms 2024-09-19T06:29:45.711 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1687873 13.29 MB/sec execute 569 sec latency 53.465 ms 2024-09-19T06:29:46.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:46 smithi053 ceph-mon[27968]: pgmap v608: 129 pgs: 129 active+clean; 212 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 MiB/s wr, 87 op/s 2024-09-19T06:29:46.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:46 smithi155 ceph-mon[35292]: pgmap v608: 129 pgs: 129 active+clean; 212 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 MiB/s wr, 87 op/s 2024-09-19T06:29:46.635 INFO:tasks.workunit.client.1.smithi155.stdout: 1 522686 4.09 MB/sec execute 573 sec latency 25.835 ms 2024-09-19T06:29:46.711 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1687995 13.27 MB/sec execute 570 sec latency 275.141 ms 2024-09-19T06:29:47.635 INFO:tasks.workunit.client.1.smithi155.stdout: 1 523347 4.09 MB/sec execute 574 sec latency 56.643 ms 2024-09-19T06:29:47.711 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1688074 13.25 MB/sec execute 571 sec latency 320.496 ms 2024-09-19T06:29:48.346 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:48 smithi053 ceph-mon[27968]: pgmap v609: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 87 op/s 2024-09-19T06:29:48.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:48 smithi155 ceph-mon[35292]: pgmap v609: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 87 op/s 2024-09-19T06:29:48.635 INFO:tasks.workunit.client.1.smithi155.stdout: 1 524052 4.09 MB/sec execute 575 sec latency 126.035 ms 2024-09-19T06:29:48.711 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1690524 13.25 MB/sec execute 572 sec latency 275.016 ms 2024-09-19T06:29:49.635 INFO:tasks.workunit.client.1.smithi155.stdout: 1 524814 4.09 MB/sec execute 576 sec latency 12.353 ms 2024-09-19T06:29:49.711 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1694029 13.26 MB/sec execute 573 sec latency 33.425 ms 2024-09-19T06:29:50.356 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:50 smithi053 ceph-mon[27968]: pgmap v610: 129 pgs: 129 active+clean; 212 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.1 MiB/s wr, 96 op/s 2024-09-19T06:29:50.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:50 smithi155 ceph-mon[35292]: pgmap v610: 129 pgs: 129 active+clean; 212 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.1 MiB/s wr, 96 op/s 2024-09-19T06:29:50.635 INFO:tasks.workunit.client.1.smithi155.stdout: 1 525605 4.09 MB/sec execute 577 sec latency 39.907 ms 2024-09-19T06:29:50.711 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1695918 13.25 MB/sec execute 574 sec latency 207.748 ms 2024-09-19T06:29:51.635 INFO:tasks.workunit.client.1.smithi155.stdout: 1 526278 4.09 MB/sec execute 578 sec latency 12.701 ms 2024-09-19T06:29:51.711 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1698571 13.26 MB/sec execute 575 sec latency 358.862 ms 2024-09-19T06:29:52.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:52 smithi053 ceph-mon[27968]: pgmap v611: 129 pgs: 129 active+clean; 212 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.6 MiB/s wr, 80 op/s 2024-09-19T06:29:52.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:52 smithi155 ceph-mon[35292]: pgmap v611: 129 pgs: 129 active+clean; 212 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.6 MiB/s wr, 80 op/s 2024-09-19T06:29:52.635 INFO:tasks.workunit.client.1.smithi155.stdout: 1 527048 4.09 MB/sec execute 579 sec latency 15.511 ms 2024-09-19T06:29:52.711 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1702051 13.27 MB/sec execute 576 sec latency 50.126 ms 2024-09-19T06:29:53.635 INFO:tasks.workunit.client.1.smithi155.stdout: 1 527913 4.09 MB/sec execute 580 sec latency 21.121 ms 2024-09-19T06:29:53.711 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1704672 13.27 MB/sec execute 577 sec latency 33.498 ms 2024-09-19T06:29:54.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:54 smithi053 ceph-mon[27968]: pgmap v612: 129 pgs: 129 active+clean; 213 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.1 MiB/s wr, 104 op/s 2024-09-19T06:29:54.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:54 smithi155 ceph-mon[35292]: pgmap v612: 129 pgs: 129 active+clean; 213 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.1 MiB/s wr, 104 op/s 2024-09-19T06:29:54.635 INFO:tasks.workunit.client.1.smithi155.stdout: 1 528778 4.09 MB/sec execute 581 sec latency 46.021 ms 2024-09-19T06:29:54.711 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1705655 13.25 MB/sec execute 578 sec latency 287.374 ms 2024-09-19T06:29:55.635 INFO:tasks.workunit.client.1.smithi155.stdout: 1 529509 4.10 MB/sec execute 582 sec latency 47.299 ms 2024-09-19T06:29:55.711 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1705723 13.23 MB/sec execute 579 sec latency 275.492 ms 2024-09-19T06:29:56.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:56 smithi053 ceph-mon[27968]: pgmap v613: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.5 MiB/s wr, 75 op/s 2024-09-19T06:29:56.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:56 smithi155 ceph-mon[35292]: pgmap v613: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.5 MiB/s wr, 75 op/s 2024-09-19T06:29:56.635 INFO:tasks.workunit.client.1.smithi155.stdout: 1 530029 4.09 MB/sec execute 583 sec latency 15.923 ms 2024-09-19T06:29:56.711 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1709195 13.24 MB/sec execute 580 sec latency 42.035 ms 2024-09-19T06:29:57.635 INFO:tasks.workunit.client.1.smithi155.stdout: 1 530844 4.09 MB/sec execute 584 sec latency 11.180 ms 2024-09-19T06:29:57.711 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1711226 13.24 MB/sec execute 581 sec latency 312.703 ms 2024-09-19T06:29:58.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:58 smithi053 ceph-mon[27968]: pgmap v614: 129 pgs: 129 active+clean; 215 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.6 MiB/s wr, 93 op/s 2024-09-19T06:29:58.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:58 smithi155 ceph-mon[35292]: pgmap v614: 129 pgs: 129 active+clean; 215 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.6 MiB/s wr, 93 op/s 2024-09-19T06:29:58.635 INFO:tasks.workunit.client.1.smithi155.stdout: 1 531719 4.09 MB/sec execute 585 sec latency 24.147 ms 2024-09-19T06:29:58.711 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1713648 13.24 MB/sec execute 582 sec latency 373.216 ms 2024-09-19T06:29:59.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:29:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:29:59.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:29:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:29:59.635 INFO:tasks.workunit.client.1.smithi155.stdout: 1 532650 4.09 MB/sec execute 586 sec latency 15.632 ms 2024-09-19T06:29:59.711 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1716474 13.24 MB/sec execute 583 sec latency 33.485 ms 2024-09-19T06:30:00.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:00 smithi053 ceph-mon[27968]: pgmap v615: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.1 MiB/s wr, 100 op/s 2024-09-19T06:30:00.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:00 smithi053 ceph-mon[27968]: overall HEALTH_OK 2024-09-19T06:30:00.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:00 smithi155 ceph-mon[35292]: pgmap v615: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.1 MiB/s wr, 100 op/s 2024-09-19T06:30:00.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:00 smithi155 ceph-mon[35292]: overall HEALTH_OK 2024-09-19T06:30:00.635 INFO:tasks.workunit.client.1.smithi155.stdout: 1 533287 4.09 MB/sec execute 587 sec latency 12.432 ms 2024-09-19T06:30:00.711 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1719912 13.25 MB/sec execute 584 sec latency 40.985 ms 2024-09-19T06:30:01.635 INFO:tasks.workunit.client.1.smithi155.stdout: 1 533856 4.09 MB/sec execute 588 sec latency 50.274 ms 2024-09-19T06:30:01.711 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1720089 13.23 MB/sec execute 585 sec latency 356.452 ms 2024-09-19T06:30:02.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:02 smithi053 ceph-mon[27968]: pgmap v616: 129 pgs: 129 active+clean; 215 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 74 op/s 2024-09-19T06:30:02.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:02 smithi155 ceph-mon[35292]: pgmap v616: 129 pgs: 129 active+clean; 215 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 74 op/s 2024-09-19T06:30:02.635 INFO:tasks.workunit.client.1.smithi155.stdout: 1 534694 4.09 MB/sec execute 589 sec latency 25.801 ms 2024-09-19T06:30:02.711 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1722078 13.22 MB/sec execute 586 sec latency 274.574 ms 2024-09-19T06:30:03.635 INFO:tasks.workunit.client.1.smithi155.stdout: 1 535477 4.09 MB/sec execute 590 sec latency 14.615 ms 2024-09-19T06:30:03.711 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1725060 13.22 MB/sec execute 587 sec latency 33.492 ms 2024-09-19T06:30:04.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:04 smithi053 ceph-mon[27968]: pgmap v617: 129 pgs: 129 active+clean; 197 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.1 MiB/s wr, 107 op/s 2024-09-19T06:30:04.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:04 smithi155 ceph-mon[35292]: pgmap v617: 129 pgs: 129 active+clean; 197 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.1 MiB/s wr, 107 op/s 2024-09-19T06:30:04.635 INFO:tasks.workunit.client.1.smithi155.stdout: 1 536259 4.09 MB/sec execute 591 sec latency 14.339 ms 2024-09-19T06:30:04.712 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1728080 13.23 MB/sec execute 588 sec latency 41.646 ms 2024-09-19T06:30:05.636 INFO:tasks.workunit.client.1.smithi155.stdout: 1 536932 4.09 MB/sec execute 592 sec latency 10.120 ms 2024-09-19T06:30:05.712 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1730545 13.23 MB/sec execute 589 sec latency 58.227 ms 2024-09-19T06:30:06.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:06 smithi053 ceph-mon[27968]: pgmap v618: 129 pgs: 129 active+clean; 196 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 83 op/s 2024-09-19T06:30:06.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:06 smithi155 ceph-mon[35292]: pgmap v618: 129 pgs: 129 active+clean; 196 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.6 MiB/s wr, 83 op/s 2024-09-19T06:30:06.635 INFO:tasks.workunit.client.1.smithi155.stdout: 1 537670 4.09 MB/sec execute 593 sec latency 16.719 ms 2024-09-19T06:30:06.712 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1734019 13.24 MB/sec execute 590 sec latency 79.079 ms 2024-09-19T06:30:07.636 INFO:tasks.workunit.client.1.smithi155.stdout: 1 538530 4.09 MB/sec execute 594 sec latency 14.784 ms 2024-09-19T06:30:07.712 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1736475 13.24 MB/sec execute 591 sec latency 280.955 ms 2024-09-19T06:30:08.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:08 smithi053 ceph-mon[27968]: pgmap v619: 129 pgs: 129 active+clean; 197 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 MiB/s wr, 96 op/s 2024-09-19T06:30:08.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:08 smithi155 ceph-mon[35292]: pgmap v619: 129 pgs: 129 active+clean; 197 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 MiB/s wr, 96 op/s 2024-09-19T06:30:08.635 INFO:tasks.workunit.client.1.smithi155.stdout: 1 539403 4.09 MB/sec execute 595 sec latency 41.468 ms 2024-09-19T06:30:08.712 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1736616 13.22 MB/sec execute 592 sec latency 282.772 ms 2024-09-19T06:30:09.635 INFO:tasks.workunit.client.1.smithi155.stdout: 1 540178 4.10 MB/sec execute 596 sec latency 12.399 ms 2024-09-19T06:30:09.712 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1739302 13.22 MB/sec execute 593 sec latency 224.906 ms 2024-09-19T06:30:10.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:10 smithi053 ceph-mon[27968]: pgmap v620: 129 pgs: 129 active+clean; 197 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.1 MiB/s wr, 94 op/s 2024-09-19T06:30:10.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:10 smithi155 ceph-mon[35292]: pgmap v620: 129 pgs: 129 active+clean; 197 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.1 MiB/s wr, 94 op/s 2024-09-19T06:30:10.636 INFO:tasks.workunit.client.1.smithi155.stdout: 1 540701 4.09 MB/sec execute 597 sec latency 15.389 ms 2024-09-19T06:30:10.712 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1742718 13.23 MB/sec execute 594 sec latency 33.292 ms 2024-09-19T06:30:11.636 INFO:tasks.workunit.client.1.smithi155.stdout: 1 541512 4.09 MB/sec execute 598 sec latency 14.650 ms 2024-09-19T06:30:11.712 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1744799 13.23 MB/sec execute 595 sec latency 150.359 ms 2024-09-19T06:30:12.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:12 smithi053 ceph-mon[27968]: pgmap v621: 129 pgs: 129 active+clean; 198 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.7 MiB/s wr, 85 op/s 2024-09-19T06:30:12.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:12 smithi155 ceph-mon[35292]: pgmap v621: 129 pgs: 129 active+clean; 198 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.7 MiB/s wr, 85 op/s 2024-09-19T06:30:12.636 INFO:tasks.workunit.client.1.smithi155.stdout: 1 542357 4.09 MB/sec execute 599 sec latency 22.673 ms 2024-09-19T06:30:12.712 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1748208 13.24 MB/sec execute 596 sec latency 33.472 ms 2024-09-19T06:30:13.636 INFO:tasks.workunit.client.1.smithi155.stdout: 1 cleanup 600 sec 2024-09-19T06:30:13.712 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1751690 13.25 MB/sec execute 597 sec latency 50.100 ms 2024-09-19T06:30:13.919 INFO:tasks.workunit.client.1.smithi155.stdout: 0 cleanup 600 sec 2024-09-19T06:30:13.919 INFO:tasks.workunit.client.1.smithi155.stdout: 2024-09-19T06:30:13.919 INFO:tasks.workunit.client.1.smithi155.stdout: Operation Count AvgLat MaxLat 2024-09-19T06:30:13.919 INFO:tasks.workunit.client.1.smithi155.stdout: ---------------------------------------- 2024-09-19T06:30:13.919 INFO:tasks.workunit.client.1.smithi155.stdout: NTCreateX 78066 2.158 100.824 2024-09-19T06:30:13.920 INFO:tasks.workunit.client.1.smithi155.stdout: Close 57338 0.042 1.667 2024-09-19T06:30:13.920 INFO:tasks.workunit.client.1.smithi155.stdout: Rename 3309 6.136 100.994 2024-09-19T06:30:13.920 INFO:tasks.workunit.client.1.smithi155.stdout: Unlink 15762 2.514 94.756 2024-09-19T06:30:13.920 INFO:tasks.workunit.client.1.smithi155.stdout: Deltree 2 110.388 176.107 2024-09-19T06:30:13.920 INFO:tasks.workunit.client.1.smithi155.stdout: Mkdir 1 0.229 0.229 2024-09-19T06:30:13.920 INFO:tasks.workunit.client.1.smithi155.stdout: Qpathinfo 70762 1.302 126.029 2024-09-19T06:30:13.920 INFO:tasks.workunit.client.1.smithi155.stdout: Qfileinfo 12415 0.065 1.289 2024-09-19T06:30:13.920 INFO:tasks.workunit.client.1.smithi155.stdout: Qfsinfo 12990 0.285 25.798 2024-09-19T06:30:13.920 INFO:tasks.workunit.client.1.smithi155.stdout: Sfileinfo 6354 0.534 3.510 2024-09-19T06:30:13.920 INFO:tasks.workunit.client.1.smithi155.stdout: Find 27372 9.070 193.130 2024-09-19T06:30:13.920 INFO:tasks.workunit.client.1.smithi155.stdout: WriteX 38953 0.145 1.546 2024-09-19T06:30:13.920 INFO:tasks.workunit.client.1.smithi155.stdout: ReadX 122652 0.003 0.803 2024-09-19T06:30:13.920 INFO:tasks.workunit.client.1.smithi155.stdout: LockX 256 0.613 4.298 2024-09-19T06:30:13.921 INFO:tasks.workunit.client.1.smithi155.stdout: UnlockX 256 0.448 1.541 2024-09-19T06:30:13.921 INFO:tasks.workunit.client.1.smithi155.stdout: Flush 5474 2.426 43.062 2024-09-19T06:30:13.921 INFO:tasks.workunit.client.1.smithi155.stdout: 2024-09-19T06:30:13.921 INFO:tasks.workunit.client.1.smithi155.stdout:Throughput 4.08799 MB/sec 1 clients 1 procs max_latency=193.136 ms 2024-09-19T06:30:13.921 INFO:tasks.workunit.client.1.smithi155.stderr:+ dbench 10 2024-09-19T06:30:13.922 INFO:tasks.workunit.client.1.smithi155.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-19T06:30:13.922 INFO:tasks.workunit.client.1.smithi155.stdout: 2024-09-19T06:30:13.922 INFO:tasks.workunit.client.1.smithi155.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-19T06:30:13.922 INFO:tasks.workunit.client.1.smithi155.stdout:9 of 10 processes prepared for launch 0 sec 2024-09-19T06:30:14.021 INFO:tasks.workunit.client.1.smithi155.stdout:10 of 10 processes prepared for launch 0 sec 2024-09-19T06:30:14.021 INFO:tasks.workunit.client.1.smithi155.stdout:releasing clients 2024-09-19T06:30:14.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:14 smithi053 ceph-mon[27968]: pgmap v622: 129 pgs: 129 active+clean; 198 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.2 MiB/s wr, 116 op/s 2024-09-19T06:30:14.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:14 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:30:14.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:14 smithi155 ceph-mon[35292]: pgmap v622: 129 pgs: 129 active+clean; 198 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.1 KiB/s rd, 2.2 MiB/s wr, 116 op/s 2024-09-19T06:30:14.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:14 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:30:14.712 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1755271 13.26 MB/sec execute 598 sec latency 33.239 ms 2024-09-19T06:30:15.021 INFO:tasks.workunit.client.1.smithi155.stdout: 10 314 115.11 MB/sec warmup 1 sec latency 33.710 ms 2024-09-19T06:30:15.712 INFO:tasks.workunit.client.0.smithi053.stdout: 1 1757789 13.26 MB/sec execute 599 sec latency 40.614 ms 2024-09-19T06:30:16.021 INFO:tasks.workunit.client.1.smithi155.stdout: 10 599 103.92 MB/sec warmup 2 sec latency 119.317 ms 2024-09-19T06:30:16.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:16 smithi053 ceph-mon[27968]: pgmap v623: 129 pgs: 129 active+clean; 208 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 2.5 MiB/s wr, 87 op/s 2024-09-19T06:30:16.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:16 smithi155 ceph-mon[35292]: pgmap v623: 129 pgs: 129 active+clean; 208 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 2.5 MiB/s wr, 87 op/s 2024-09-19T06:30:16.712 INFO:tasks.workunit.client.0.smithi053.stdout: 1 cleanup 600 sec 2024-09-19T06:30:16.735 INFO:tasks.workunit.client.0.smithi053.stdout: 0 cleanup 600 sec 2024-09-19T06:30:16.735 INFO:tasks.workunit.client.0.smithi053.stdout: 2024-09-19T06:30:16.736 INFO:tasks.workunit.client.0.smithi053.stdout: Operation Count AvgLat MaxLat 2024-09-19T06:30:16.736 INFO:tasks.workunit.client.0.smithi053.stdout: ---------------------------------------- 2024-09-19T06:30:16.736 INFO:tasks.workunit.client.0.smithi053.stdout: NTCreateX 253249 0.011 87.754 2024-09-19T06:30:16.736 INFO:tasks.workunit.client.0.smithi053.stdout: Close 185991 0.001 0.069 2024-09-19T06:30:16.736 INFO:tasks.workunit.client.0.smithi053.stdout: Rename 10720 0.024 0.106 2024-09-19T06:30:16.736 INFO:tasks.workunit.client.0.smithi053.stdout: Unlink 51177 0.021 109.123 2024-09-19T06:30:16.736 INFO:tasks.workunit.client.0.smithi053.stdout: Deltree 6 1.692 3.493 2024-09-19T06:30:16.736 INFO:tasks.workunit.client.0.smithi053.stdout: Mkdir 3 0.003 0.003 2024-09-19T06:30:16.736 INFO:tasks.workunit.client.0.smithi053.stdout: Qpathinfo 229521 0.005 0.116 2024-09-19T06:30:16.736 INFO:tasks.workunit.client.0.smithi053.stdout: Qfileinfo 40191 0.001 0.069 2024-09-19T06:30:16.736 INFO:tasks.workunit.client.0.smithi053.stdout: Qfsinfo 42096 0.002 0.032 2024-09-19T06:30:16.736 INFO:tasks.workunit.client.0.smithi053.stdout: Sfileinfo 20616 0.006 0.071 2024-09-19T06:30:16.736 INFO:tasks.workunit.client.0.smithi053.stdout: Find 88740 0.017 0.129 2024-09-19T06:30:16.737 INFO:tasks.workunit.client.0.smithi053.stdout: WriteX 126141 0.013 0.794 2024-09-19T06:30:16.737 INFO:tasks.workunit.client.0.smithi053.stdout: ReadX 396850 0.002 64.855 2024-09-19T06:30:16.737 INFO:tasks.workunit.client.0.smithi053.stdout: LockX 824 0.003 0.016 2024-09-19T06:30:16.737 INFO:tasks.workunit.client.0.smithi053.stdout: UnlockX 824 0.002 0.005 2024-09-19T06:30:16.737 INFO:tasks.workunit.client.0.smithi053.stdout: Flush 17744 33.143 593.751 2024-09-19T06:30:16.737 INFO:tasks.workunit.client.0.smithi053.stdout: 2024-09-19T06:30:16.737 INFO:tasks.workunit.client.0.smithi053.stdout:Throughput 13.2569 MB/sec 1 clients 1 procs max_latency=593.758 ms 2024-09-19T06:30:16.737 INFO:tasks.workunit.client.0.smithi053.stderr:+ dbench 10 2024-09-19T06:30:16.738 INFO:tasks.workunit.client.0.smithi053.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-19T06:30:16.738 INFO:tasks.workunit.client.0.smithi053.stdout: 2024-09-19T06:30:16.738 INFO:tasks.workunit.client.0.smithi053.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-19T06:30:16.738 INFO:tasks.workunit.client.0.smithi053.stdout:8 of 10 processes prepared for launch 0 sec 2024-09-19T06:30:16.837 INFO:tasks.workunit.client.0.smithi053.stdout:10 of 10 processes prepared for launch 0 sec 2024-09-19T06:30:16.837 INFO:tasks.workunit.client.0.smithi053.stdout:releasing clients 2024-09-19T06:30:17.021 INFO:tasks.workunit.client.1.smithi155.stdout: 10 749 71.11 MB/sec warmup 3 sec latency 231.734 ms 2024-09-19T06:30:17.839 INFO:tasks.workunit.client.0.smithi053.stdout: 10 669 211.30 MB/sec warmup 1 sec latency 417.549 ms 2024-09-19T06:30:18.021 INFO:tasks.workunit.client.1.smithi155.stdout: 10 906 54.31 MB/sec warmup 4 sec latency 139.684 ms 2024-09-19T06:30:18.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:18 smithi053 ceph-mon[27968]: pgmap v624: 129 pgs: 129 active+clean; 255 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 3.4 KiB/s rd, 6.8 MiB/s wr, 102 op/s 2024-09-19T06:30:18.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:18 smithi155 ceph-mon[35292]: pgmap v624: 129 pgs: 129 active+clean; 255 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 3.4 KiB/s rd, 6.8 MiB/s wr, 102 op/s 2024-09-19T06:30:18.841 INFO:tasks.workunit.client.0.smithi053.stdout: 10 676 105.80 MB/sec warmup 2 sec latency 470.198 ms 2024-09-19T06:30:19.021 INFO:tasks.workunit.client.1.smithi155.stdout: 10 1039 45.78 MB/sec warmup 5 sec latency 157.517 ms 2024-09-19T06:30:19.839 INFO:tasks.workunit.client.0.smithi053.stdout: 10 726 70.97 MB/sec warmup 3 sec latency 392.675 ms 2024-09-19T06:30:20.022 INFO:tasks.workunit.client.1.smithi155.stdout: 10 1188 38.39 MB/sec warmup 6 sec latency 95.193 ms 2024-09-19T06:30:20.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:20 smithi053 ceph-mon[27968]: pgmap v625: 129 pgs: 129 active+clean; 363 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 4.7 KiB/s rd, 17 MiB/s wr, 149 op/s 2024-09-19T06:30:20.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:20 smithi155 ceph-mon[35292]: pgmap v625: 129 pgs: 129 active+clean; 363 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 4.7 KiB/s rd, 17 MiB/s wr, 149 op/s 2024-09-19T06:30:20.839 INFO:tasks.workunit.client.0.smithi053.stdout: 10 733 53.26 MB/sec warmup 4 sec latency 476.826 ms 2024-09-19T06:30:21.022 INFO:tasks.workunit.client.1.smithi155.stdout: 10 1328 35.10 MB/sec warmup 7 sec latency 197.898 ms 2024-09-19T06:30:21.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:21 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:30:21.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:21 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:30:21.839 INFO:tasks.workunit.client.0.smithi053.stdout: 10 794 42.92 MB/sec warmup 5 sec latency 346.826 ms 2024-09-19T06:30:22.022 INFO:tasks.workunit.client.1.smithi155.stdout: 10 1464 32.54 MB/sec warmup 8 sec latency 193.874 ms 2024-09-19T06:30:22.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:22 smithi053 ceph-mon[27968]: pgmap v626: 129 pgs: 129 active+clean; 384 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 2.9 KiB/s rd, 18 MiB/s wr, 138 op/s 2024-09-19T06:30:22.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:22 smithi155 ceph-mon[35292]: pgmap v626: 129 pgs: 129 active+clean; 384 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 2.9 KiB/s rd, 18 MiB/s wr, 138 op/s 2024-09-19T06:30:22.839 INFO:tasks.workunit.client.0.smithi053.stdout: 10 808 35.82 MB/sec warmup 6 sec latency 308.855 ms 2024-09-19T06:30:23.022 INFO:tasks.workunit.client.1.smithi155.stdout: 10 1576 29.03 MB/sec warmup 9 sec latency 141.761 ms 2024-09-19T06:30:23.839 INFO:tasks.workunit.client.0.smithi053.stdout: 10 916 31.09 MB/sec warmup 7 sec latency 312.027 ms 2024-09-19T06:30:24.022 INFO:tasks.workunit.client.1.smithi155.stdout: 10 1682 26.22 MB/sec warmup 10 sec latency 144.589 ms 2024-09-19T06:30:24.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:24 smithi053 ceph-mon[27968]: pgmap v627: 129 pgs: 129 active+clean; 385 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 4.9 KiB/s rd, 19 MiB/s wr, 202 op/s 2024-09-19T06:30:24.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:24 smithi155 ceph-mon[35292]: pgmap v627: 129 pgs: 129 active+clean; 385 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 4.9 KiB/s rd, 19 MiB/s wr, 202 op/s 2024-09-19T06:30:24.839 INFO:tasks.workunit.client.0.smithi053.stdout: 10 1496 32.56 MB/sec warmup 8 sec latency 285.518 ms 2024-09-19T06:30:25.022 INFO:tasks.workunit.client.1.smithi155.stdout: 10 1866 23.97 MB/sec warmup 11 sec latency 129.503 ms 2024-09-19T06:30:25.839 INFO:tasks.workunit.client.0.smithi053.stdout: 10 1886 29.30 MB/sec warmup 9 sec latency 316.025 ms 2024-09-19T06:30:26.022 INFO:tasks.workunit.client.1.smithi155.stdout: 10 2013 22.11 MB/sec warmup 12 sec latency 155.209 ms 2024-09-19T06:30:26.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:26 smithi053 ceph-mon[27968]: pgmap v628: 129 pgs: 129 active+clean; 384 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 3.3 KiB/s rd, 19 MiB/s wr, 174 op/s 2024-09-19T06:30:26.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:26 smithi155 ceph-mon[35292]: pgmap v628: 129 pgs: 129 active+clean; 384 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 3.3 KiB/s rd, 19 MiB/s wr, 174 op/s 2024-09-19T06:30:26.839 INFO:tasks.workunit.client.0.smithi053.stdout: 10 2380 27.97 MB/sec warmup 10 sec latency 433.097 ms 2024-09-19T06:30:27.022 INFO:tasks.workunit.client.1.smithi155.stdout: 10 2146 20.54 MB/sec warmup 13 sec latency 126.955 ms 2024-09-19T06:30:27.839 INFO:tasks.workunit.client.0.smithi053.stdout: 10 2783 29.19 MB/sec warmup 11 sec latency 355.453 ms 2024-09-19T06:30:28.022 INFO:tasks.workunit.client.1.smithi155.stdout: 10 2329 19.87 MB/sec warmup 14 sec latency 132.477 ms 2024-09-19T06:30:28.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:28 smithi053 ceph-mon[27968]: pgmap v629: 129 pgs: 129 active+clean; 386 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 3.3 KiB/s rd, 18 MiB/s wr, 203 op/s 2024-09-19T06:30:28.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:28 smithi155 ceph-mon[35292]: pgmap v629: 129 pgs: 129 active+clean; 386 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 3.3 KiB/s rd, 18 MiB/s wr, 203 op/s 2024-09-19T06:30:28.839 INFO:tasks.workunit.client.0.smithi053.stdout: 10 2983 27.93 MB/sec warmup 12 sec latency 564.985 ms 2024-09-19T06:30:29.022 INFO:tasks.workunit.client.1.smithi155.stdout: 10 2406 18.68 MB/sec warmup 15 sec latency 260.015 ms 2024-09-19T06:30:29.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:30:29.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:29 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:30:29.839 INFO:tasks.workunit.client.0.smithi053.stdout: 10 3297 27.19 MB/sec warmup 13 sec latency 640.257 ms 2024-09-19T06:30:30.022 INFO:tasks.workunit.client.1.smithi155.stdout: 10 2539 18.04 MB/sec warmup 16 sec latency 96.169 ms 2024-09-19T06:30:30.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:30 smithi053 ceph-mon[27968]: pgmap v630: 129 pgs: 129 active+clean; 387 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 4.8 KiB/s rd, 14 MiB/s wr, 211 op/s 2024-09-19T06:30:30.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:30 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:30:30.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:30 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:30:30.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:30 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:30:30.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:30 smithi155 ceph-mon[35292]: pgmap v630: 129 pgs: 129 active+clean; 387 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 4.8 KiB/s rd, 14 MiB/s wr, 211 op/s 2024-09-19T06:30:30.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:30 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:30:30.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:30 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:30:30.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:30 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:30:30.840 INFO:tasks.workunit.client.0.smithi053.stdout: 10 3339 25.27 MB/sec warmup 14 sec latency 375.437 ms 2024-09-19T06:30:31.022 INFO:tasks.workunit.client.1.smithi155.stdout: 10 2637 17.43 MB/sec warmup 17 sec latency 151.815 ms 2024-09-19T06:30:31.839 INFO:tasks.workunit.client.0.smithi053.stdout: 10 4238 26.85 MB/sec warmup 15 sec latency 450.308 ms 2024-09-19T06:30:32.022 INFO:tasks.workunit.client.1.smithi155.stdout: 10 2872 18.13 MB/sec warmup 18 sec latency 161.636 ms 2024-09-19T06:30:32.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:32 smithi053 ceph-mon[27968]: pgmap v631: 129 pgs: 129 active+clean; 387 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 3.5 KiB/s rd, 4.0 MiB/s wr, 151 op/s 2024-09-19T06:30:32.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:32 smithi155 ceph-mon[35292]: pgmap v631: 129 pgs: 129 active+clean; 387 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 3.5 KiB/s rd, 4.0 MiB/s wr, 151 op/s 2024-09-19T06:30:32.839 INFO:tasks.workunit.client.0.smithi053.stdout: 10 4245 25.18 MB/sec warmup 16 sec latency 400.871 ms 2024-09-19T06:30:33.022 INFO:tasks.workunit.client.1.smithi155.stdout: 10 3053 17.75 MB/sec warmup 19 sec latency 173.263 ms 2024-09-19T06:30:33.839 INFO:tasks.workunit.client.0.smithi053.stdout: 10 4443 23.99 MB/sec warmup 17 sec latency 300.493 ms 2024-09-19T06:30:34.022 INFO:tasks.workunit.client.1.smithi155.stdout: 10 3287 17.67 MB/sec warmup 20 sec latency 112.552 ms 2024-09-19T06:30:34.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:34 smithi053 ceph-mon[27968]: pgmap v632: 129 pgs: 129 active+clean; 398 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.1 KiB/s rd, 3.4 MiB/s wr, 183 op/s 2024-09-19T06:30:34.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:34 smithi155 ceph-mon[35292]: pgmap v632: 129 pgs: 129 active+clean; 398 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.1 KiB/s rd, 3.4 MiB/s wr, 183 op/s 2024-09-19T06:30:34.840 INFO:tasks.workunit.client.0.smithi053.stdout: 10 5197 25.16 MB/sec warmup 18 sec latency 297.444 ms 2024-09-19T06:30:35.022 INFO:tasks.workunit.client.1.smithi155.stdout: 10 3313 16.84 MB/sec warmup 21 sec latency 138.064 ms 2024-09-19T06:30:35.840 INFO:tasks.workunit.client.0.smithi053.stdout: 10 6065 25.34 MB/sec warmup 19 sec latency 450.953 ms 2024-09-19T06:30:36.022 INFO:tasks.workunit.client.1.smithi155.stdout: 10 3338 16.08 MB/sec warmup 22 sec latency 85.737 ms 2024-09-19T06:30:36.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:36 smithi053 ceph-mon[27968]: pgmap v633: 129 pgs: 129 active+clean; 405 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.1 KiB/s rd, 3.0 MiB/s wr, 123 op/s 2024-09-19T06:30:36.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:36 smithi155 ceph-mon[35292]: pgmap v633: 129 pgs: 129 active+clean; 405 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.1 KiB/s rd, 3.0 MiB/s wr, 123 op/s 2024-09-19T06:30:36.840 INFO:tasks.workunit.client.0.smithi053.stdout: 10 7778 29.70 MB/sec warmup 20 sec latency 121.879 ms 2024-09-19T06:30:37.022 INFO:tasks.workunit.client.1.smithi155.stdout: 10 3549 16.71 MB/sec warmup 23 sec latency 110.438 ms 2024-09-19T06:30:37.840 INFO:tasks.workunit.client.0.smithi053.stdout: 10 7907 28.43 MB/sec warmup 21 sec latency 101.833 ms 2024-09-19T06:30:38.022 INFO:tasks.workunit.client.1.smithi155.stdout: 10 3719 16.31 MB/sec warmup 24 sec latency 129.610 ms 2024-09-19T06:30:38.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:38 smithi053 ceph-mon[27968]: pgmap v634: 129 pgs: 129 active+clean; 411 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.1 KiB/s rd, 3.6 MiB/s wr, 138 op/s 2024-09-19T06:30:38.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:38 smithi155 ceph-mon[35292]: pgmap v634: 129 pgs: 129 active+clean; 411 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.1 KiB/s rd, 3.6 MiB/s wr, 138 op/s 2024-09-19T06:30:38.840 INFO:tasks.workunit.client.0.smithi053.stdout: 10 7921 27.15 MB/sec warmup 22 sec latency 434.218 ms 2024-09-19T06:30:39.022 INFO:tasks.workunit.client.1.smithi155.stdout: 10 3806 15.91 MB/sec warmup 25 sec latency 271.045 ms 2024-09-19T06:30:39.840 INFO:tasks.workunit.client.0.smithi053.stdout: 10 8025 26.09 MB/sec warmup 23 sec latency 376.057 ms 2024-09-19T06:30:40.022 INFO:tasks.workunit.client.1.smithi155.stdout: 10 3880 15.32 MB/sec warmup 26 sec latency 165.731 ms 2024-09-19T06:30:40.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:40 smithi053 ceph-mon[27968]: pgmap v635: 129 pgs: 129 active+clean; 413 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.4 KiB/s rd, 4.0 MiB/s wr, 115 op/s 2024-09-19T06:30:40.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:40 smithi155 ceph-mon[35292]: pgmap v635: 129 pgs: 129 active+clean; 413 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.4 KiB/s rd, 4.0 MiB/s wr, 115 op/s 2024-09-19T06:30:40.840 INFO:tasks.workunit.client.0.smithi053.stdout: 10 8091 25.06 MB/sec warmup 24 sec latency 381.302 ms 2024-09-19T06:30:41.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 3940 14.84 MB/sec warmup 27 sec latency 228.954 ms 2024-09-19T06:30:41.840 INFO:tasks.workunit.client.0.smithi053.stdout: 10 8710 25.75 MB/sec warmup 25 sec latency 493.173 ms 2024-09-19T06:30:42.022 INFO:tasks.workunit.client.1.smithi155.stdout: 10 4009 14.31 MB/sec warmup 28 sec latency 144.491 ms 2024-09-19T06:30:42.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:42 smithi053 ceph-mon[27968]: pgmap v636: 129 pgs: 129 active+clean; 414 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.9 KiB/s rd, 4.2 MiB/s wr, 90 op/s 2024-09-19T06:30:42.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:42 smithi155 ceph-mon[35292]: pgmap v636: 129 pgs: 129 active+clean; 414 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.9 KiB/s rd, 4.2 MiB/s wr, 90 op/s 2024-09-19T06:30:42.840 INFO:tasks.workunit.client.0.smithi053.stdout: 10 8943 24.83 MB/sec warmup 26 sec latency 533.935 ms 2024-09-19T06:30:43.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 4095 13.82 MB/sec warmup 29 sec latency 211.916 ms 2024-09-19T06:30:43.840 INFO:tasks.workunit.client.0.smithi053.stdout: 10 9041 23.99 MB/sec warmup 27 sec latency 569.781 ms 2024-09-19T06:30:44.022 INFO:tasks.workunit.client.1.smithi155.stdout: 10 4201 13.38 MB/sec warmup 30 sec latency 278.754 ms 2024-09-19T06:30:44.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:44 smithi155 ceph-mon[35292]: pgmap v637: 129 pgs: 129 active+clean; 420 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 6.4 KiB/s rd, 4.9 MiB/s wr, 123 op/s 2024-09-19T06:30:44.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:44 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:30:44.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:44 smithi053 ceph-mon[27968]: pgmap v637: 129 pgs: 129 active+clean; 420 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 6.4 KiB/s rd, 4.9 MiB/s wr, 123 op/s 2024-09-19T06:30:44.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:44 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:30:44.840 INFO:tasks.workunit.client.0.smithi053.stdout: 10 9477 23.64 MB/sec warmup 28 sec latency 612.595 ms 2024-09-19T06:30:45.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 4311 13.04 MB/sec warmup 31 sec latency 175.532 ms 2024-09-19T06:30:45.840 INFO:tasks.workunit.client.0.smithi053.stdout: 10 10072 24.74 MB/sec warmup 29 sec latency 369.299 ms 2024-09-19T06:30:46.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 4439 12.73 MB/sec warmup 32 sec latency 182.932 ms 2024-09-19T06:30:46.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:46 smithi155 ceph-mon[35292]: pgmap v638: 129 pgs: 129 active+clean; 420 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.8 KiB/s rd, 3.5 MiB/s wr, 82 op/s 2024-09-19T06:30:46.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:46 smithi053 ceph-mon[27968]: pgmap v638: 129 pgs: 129 active+clean; 420 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.8 KiB/s rd, 3.5 MiB/s wr, 82 op/s 2024-09-19T06:30:46.840 INFO:tasks.workunit.client.0.smithi053.stdout: 10 11346 26.18 MB/sec warmup 30 sec latency 101.474 ms 2024-09-19T06:30:47.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 4593 12.59 MB/sec warmup 33 sec latency 88.801 ms 2024-09-19T06:30:47.840 INFO:tasks.workunit.client.0.smithi053.stdout: 10 11538 25.49 MB/sec warmup 31 sec latency 84.272 ms 2024-09-19T06:30:48.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 4744 12.40 MB/sec warmup 34 sec latency 117.153 ms 2024-09-19T06:30:48.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:48 smithi155 ceph-mon[35292]: pgmap v639: 129 pgs: 129 active+clean; 415 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.8 KiB/s rd, 3.2 MiB/s wr, 102 op/s 2024-09-19T06:30:48.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:48 smithi053 ceph-mon[27968]: pgmap v639: 129 pgs: 129 active+clean; 415 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.8 KiB/s rd, 3.2 MiB/s wr, 102 op/s 2024-09-19T06:30:48.840 INFO:tasks.workunit.client.0.smithi053.stdout: 10 12639 26.26 MB/sec warmup 32 sec latency 194.935 ms 2024-09-19T06:30:49.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 4959 12.89 MB/sec warmup 35 sec latency 111.735 ms 2024-09-19T06:30:49.840 INFO:tasks.workunit.client.0.smithi053.stdout: 10 13954 28.10 MB/sec warmup 33 sec latency 119.027 ms 2024-09-19T06:30:50.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 5110 12.56 MB/sec warmup 36 sec latency 99.664 ms 2024-09-19T06:30:50.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:50 smithi155 ceph-mon[35292]: pgmap v640: 129 pgs: 129 active+clean; 404 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 9.4 KiB/s rd, 3.2 MiB/s wr, 146 op/s 2024-09-19T06:30:50.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:50 smithi053 ceph-mon[27968]: pgmap v640: 129 pgs: 129 active+clean; 404 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 9.4 KiB/s rd, 3.2 MiB/s wr, 146 op/s 2024-09-19T06:30:50.840 INFO:tasks.workunit.client.0.smithi053.stdout: 10 14934 28.76 MB/sec warmup 34 sec latency 112.479 ms 2024-09-19T06:30:51.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 5273 12.26 MB/sec warmup 37 sec latency 105.029 ms 2024-09-19T06:30:51.840 INFO:tasks.workunit.client.0.smithi053.stdout: 10 15235 28.23 MB/sec warmup 35 sec latency 88.641 ms 2024-09-19T06:30:52.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 5469 11.98 MB/sec warmup 38 sec latency 72.389 ms 2024-09-19T06:30:52.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:52 smithi155 ceph-mon[35292]: pgmap v641: 129 pgs: 129 active+clean; 399 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 8.1 KiB/s rd, 2.8 MiB/s wr, 144 op/s 2024-09-19T06:30:52.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:52 smithi053 ceph-mon[27968]: pgmap v641: 129 pgs: 129 active+clean; 399 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 8.1 KiB/s rd, 2.8 MiB/s wr, 144 op/s 2024-09-19T06:30:52.840 INFO:tasks.workunit.client.0.smithi053.stdout: 10 16796 29.58 MB/sec warmup 36 sec latency 115.457 ms 2024-09-19T06:30:53.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 5659 11.74 MB/sec warmup 39 sec latency 75.538 ms 2024-09-19T06:30:53.840 INFO:tasks.workunit.client.0.smithi053.stdout: 10 18421 31.56 MB/sec warmup 37 sec latency 184.078 ms 2024-09-19T06:30:54.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 5845 11.65 MB/sec warmup 40 sec latency 83.985 ms 2024-09-19T06:30:54.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:54 smithi155 ceph-mon[35292]: pgmap v642: 129 pgs: 129 active+clean; 401 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 3.1 MiB/s wr, 208 op/s 2024-09-19T06:30:54.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:54 smithi053 ceph-mon[27968]: pgmap v642: 129 pgs: 129 active+clean; 401 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 3.1 MiB/s wr, 208 op/s 2024-09-19T06:30:54.840 INFO:tasks.workunit.client.0.smithi053.stdout: 10 18546 30.80 MB/sec warmup 38 sec latency 94.699 ms 2024-09-19T06:30:55.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 5969 11.52 MB/sec warmup 41 sec latency 96.739 ms 2024-09-19T06:30:55.840 INFO:tasks.workunit.client.0.smithi053.stdout: 10 18673 30.10 MB/sec warmup 39 sec latency 494.821 ms 2024-09-19T06:30:56.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 6077 11.41 MB/sec warmup 42 sec latency 221.434 ms 2024-09-19T06:30:56.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:56 smithi155 ceph-mon[35292]: pgmap v643: 129 pgs: 129 active+clean; 402 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 6.6 KiB/s rd, 2.2 MiB/s wr, 183 op/s 2024-09-19T06:30:56.505 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:56 smithi053 ceph-mon[27968]: pgmap v643: 129 pgs: 129 active+clean; 402 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 6.6 KiB/s rd, 2.2 MiB/s wr, 183 op/s 2024-09-19T06:30:56.840 INFO:tasks.workunit.client.0.smithi053.stdout: 10 18833 29.58 MB/sec warmup 40 sec latency 395.052 ms 2024-09-19T06:30:57.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 6191 11.33 MB/sec warmup 43 sec latency 172.178 ms 2024-09-19T06:30:57.840 INFO:tasks.workunit.client.0.smithi053.stdout: 10 19382 29.70 MB/sec warmup 41 sec latency 508.029 ms 2024-09-19T06:30:58.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 6394 11.72 MB/sec warmup 44 sec latency 189.533 ms 2024-09-19T06:30:58.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:58 smithi155 ceph-mon[35292]: pgmap v644: 129 pgs: 129 active+clean; 404 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 6.6 KiB/s rd, 2.5 MiB/s wr, 193 op/s 2024-09-19T06:30:58.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:58 smithi053 ceph-mon[27968]: pgmap v644: 129 pgs: 129 active+clean; 404 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 6.6 KiB/s rd, 2.5 MiB/s wr, 193 op/s 2024-09-19T06:30:58.840 INFO:tasks.workunit.client.0.smithi053.stdout: 10 19635 29.04 MB/sec warmup 42 sec latency 446.779 ms 2024-09-19T06:30:59.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 6581 11.74 MB/sec warmup 45 sec latency 184.708 ms 2024-09-19T06:30:59.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:30:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:30:59.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:30:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:30:59.840 INFO:tasks.workunit.client.0.smithi053.stdout: 10 19740 28.44 MB/sec warmup 43 sec latency 568.798 ms 2024-09-19T06:31:00.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 6807 11.77 MB/sec warmup 46 sec latency 236.484 ms 2024-09-19T06:31:00.455 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:00 smithi053 ceph-mon[27968]: pgmap v645: 129 pgs: 129 active+clean; 413 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 8.2 KiB/s rd, 3.5 MiB/s wr, 203 op/s 2024-09-19T06:31:00.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:00 smithi155 ceph-mon[35292]: pgmap v645: 129 pgs: 129 active+clean; 413 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 8.2 KiB/s rd, 3.5 MiB/s wr, 203 op/s 2024-09-19T06:31:00.840 INFO:tasks.workunit.client.0.smithi053.stdout: 10 20140 28.08 MB/sec warmup 44 sec latency 917.676 ms 2024-09-19T06:31:01.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 6853 11.59 MB/sec warmup 47 sec latency 181.162 ms 2024-09-19T06:31:01.840 INFO:tasks.workunit.client.0.smithi053.stdout: 10 20241 27.60 MB/sec warmup 45 sec latency 988.485 ms 2024-09-19T06:31:02.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 6877 11.35 MB/sec warmup 48 sec latency 143.114 ms 2024-09-19T06:31:02.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:02 smithi155 ceph-mon[35292]: pgmap v646: 129 pgs: 129 active+clean; 418 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.2 MiB/s wr, 142 op/s 2024-09-19T06:31:02.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:02 smithi053 ceph-mon[27968]: pgmap v646: 129 pgs: 129 active+clean; 418 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.2 MiB/s wr, 142 op/s 2024-09-19T06:31:02.841 INFO:tasks.workunit.client.0.smithi053.stdout: 10 20587 27.81 MB/sec warmup 46 sec latency 457.082 ms 2024-09-19T06:31:03.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 6944 11.25 MB/sec warmup 49 sec latency 96.531 ms 2024-09-19T06:31:03.841 INFO:tasks.workunit.client.0.smithi053.stdout: 10 20815 27.52 MB/sec warmup 47 sec latency 541.210 ms 2024-09-19T06:31:04.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 7070 11.34 MB/sec warmup 50 sec latency 167.060 ms 2024-09-19T06:31:04.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:04 smithi155 ceph-mon[35292]: pgmap v647: 129 pgs: 129 active+clean; 406 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 5.0 KiB/s rd, 3.9 MiB/s wr, 161 op/s 2024-09-19T06:31:04.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:04 smithi053 ceph-mon[27968]: pgmap v647: 129 pgs: 129 active+clean; 406 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 5.0 KiB/s rd, 3.9 MiB/s wr, 161 op/s 2024-09-19T06:31:04.841 INFO:tasks.workunit.client.0.smithi053.stdout: 10 21080 27.29 MB/sec warmup 48 sec latency 411.005 ms 2024-09-19T06:31:05.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 7226 11.37 MB/sec warmup 51 sec latency 175.896 ms 2024-09-19T06:31:05.841 INFO:tasks.workunit.client.0.smithi053.stdout: 10 21205 26.84 MB/sec warmup 49 sec latency 433.221 ms 2024-09-19T06:31:06.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 7314 11.27 MB/sec warmup 52 sec latency 339.332 ms 2024-09-19T06:31:06.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:06 smithi155 ceph-mon[35292]: pgmap v648: 129 pgs: 129 active+clean; 403 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 3.6 MiB/s wr, 98 op/s 2024-09-19T06:31:06.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:06 smithi053 ceph-mon[27968]: pgmap v648: 129 pgs: 129 active+clean; 403 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 3.6 MiB/s wr, 98 op/s 2024-09-19T06:31:06.841 INFO:tasks.workunit.client.0.smithi053.stdout: 10 22003 27.19 MB/sec warmup 50 sec latency 502.185 ms 2024-09-19T06:31:07.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 7393 11.12 MB/sec warmup 53 sec latency 205.533 ms 2024-09-19T06:31:07.841 INFO:tasks.workunit.client.0.smithi053.stdout: 10 22127 26.71 MB/sec warmup 51 sec latency 434.746 ms 2024-09-19T06:31:08.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 7490 10.96 MB/sec warmup 54 sec latency 140.682 ms 2024-09-19T06:31:08.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:08 smithi155 ceph-mon[35292]: pgmap v649: 129 pgs: 129 active+clean; 406 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 4.2 MiB/s wr, 114 op/s 2024-09-19T06:31:08.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:08 smithi053 ceph-mon[27968]: pgmap v649: 129 pgs: 129 active+clean; 406 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 4.2 MiB/s wr, 114 op/s 2024-09-19T06:31:08.841 INFO:tasks.workunit.client.0.smithi053.stdout: 10 23014 27.11 MB/sec warmup 52 sec latency 91.692 ms 2024-09-19T06:31:09.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 7588 10.76 MB/sec warmup 55 sec latency 118.086 ms 2024-09-19T06:31:09.841 INFO:tasks.workunit.client.0.smithi053.stdout: 10 24540 28.16 MB/sec warmup 53 sec latency 110.885 ms 2024-09-19T06:31:10.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 7702 10.58 MB/sec warmup 56 sec latency 115.972 ms 2024-09-19T06:31:10.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:10 smithi155 ceph-mon[35292]: pgmap v650: 129 pgs: 129 active+clean; 408 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 5.7 KiB/s rd, 4.4 MiB/s wr, 112 op/s 2024-09-19T06:31:10.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:10 smithi053 ceph-mon[27968]: pgmap v650: 129 pgs: 129 active+clean; 408 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 5.7 KiB/s rd, 4.4 MiB/s wr, 112 op/s 2024-09-19T06:31:10.841 INFO:tasks.workunit.client.0.smithi053.stdout: 10 25609 28.74 MB/sec warmup 54 sec latency 115.711 ms 2024-09-19T06:31:11.024 INFO:tasks.workunit.client.1.smithi155.stdout: 10 7829 10.44 MB/sec warmup 57 sec latency 176.166 ms 2024-09-19T06:31:11.841 INFO:tasks.workunit.client.0.smithi053.stdout: 10 25762 28.28 MB/sec warmup 55 sec latency 165.655 ms 2024-09-19T06:31:12.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 7978 10.32 MB/sec warmup 58 sec latency 97.147 ms 2024-09-19T06:31:12.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:12 smithi155 ceph-mon[35292]: pgmap v651: 129 pgs: 129 active+clean; 409 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 3.4 MiB/s wr, 77 op/s 2024-09-19T06:31:12.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:12 smithi053 ceph-mon[27968]: pgmap v651: 129 pgs: 129 active+clean; 409 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 3.4 MiB/s wr, 77 op/s 2024-09-19T06:31:12.841 INFO:tasks.workunit.client.0.smithi053.stdout: 10 27231 28.90 MB/sec warmup 56 sec latency 110.970 ms 2024-09-19T06:31:13.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 8119 10.25 MB/sec warmup 59 sec latency 118.806 ms 2024-09-19T06:31:13.841 INFO:tasks.workunit.client.0.smithi053.stdout: 10 28289 29.78 MB/sec warmup 57 sec latency 128.598 ms 2024-09-19T06:31:14.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 8281 10.21 MB/sec warmup 60 sec latency 88.206 ms 2024-09-19T06:31:14.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:14 smithi155 ceph-mon[35292]: pgmap v652: 129 pgs: 129 active+clean; 404 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 9.2 KiB/s rd, 3.7 MiB/s wr, 154 op/s 2024-09-19T06:31:14.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:14 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:31:14.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:14 smithi053 ceph-mon[27968]: pgmap v652: 129 pgs: 129 active+clean; 404 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 9.2 KiB/s rd, 3.7 MiB/s wr, 154 op/s 2024-09-19T06:31:14.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:14 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:31:14.841 INFO:tasks.workunit.client.0.smithi053.stdout: 10 29164 30.06 MB/sec warmup 58 sec latency 93.020 ms 2024-09-19T06:31:15.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 8455 10.30 MB/sec warmup 61 sec latency 107.554 ms 2024-09-19T06:31:15.841 INFO:tasks.workunit.client.0.smithi053.stdout: 10 29935 30.35 MB/sec warmup 59 sec latency 111.359 ms 2024-09-19T06:31:16.023 INFO:tasks.workunit.client.1.smithi155.stdout: 10 8635 10.37 MB/sec warmup 62 sec latency 99.547 ms 2024-09-19T06:31:16.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:16 smithi053 ceph-mon[27968]: pgmap v653: 129 pgs: 129 active+clean; 395 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 7.8 KiB/s rd, 3.2 MiB/s wr, 144 op/s 2024-09-19T06:31:16.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:16 smithi155 ceph-mon[35292]: pgmap v653: 129 pgs: 129 active+clean; 395 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 7.8 KiB/s rd, 3.2 MiB/s wr, 144 op/s 2024-09-19T06:31:16.841 INFO:tasks.workunit.client.0.smithi053.stdout: 10 30968 30.42 MB/sec warmup 60 sec latency 207.013 ms 2024-09-19T06:31:17.024 INFO:tasks.workunit.client.1.smithi155.stdout: 10 8785 10.23 MB/sec warmup 63 sec latency 149.546 ms 2024-09-19T06:31:17.841 INFO:tasks.workunit.client.0.smithi053.stdout: 10 31816 30.96 MB/sec warmup 61 sec latency 303.034 ms 2024-09-19T06:31:18.024 INFO:tasks.workunit.client.1.smithi155.stdout: 10 8952 10.09 MB/sec warmup 64 sec latency 160.478 ms 2024-09-19T06:31:18.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:18 smithi155 ceph-mon[35292]: pgmap v654: 129 pgs: 129 active+clean; 390 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 7.8 KiB/s rd, 3.2 MiB/s wr, 154 op/s 2024-09-19T06:31:18.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:18 smithi053 ceph-mon[27968]: pgmap v654: 129 pgs: 129 active+clean; 390 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 7.8 KiB/s rd, 3.2 MiB/s wr, 154 op/s 2024-09-19T06:31:18.841 INFO:tasks.workunit.client.0.smithi053.stdout: 10 32641 31.18 MB/sec warmup 62 sec latency 458.042 ms 2024-09-19T06:31:19.024 INFO:tasks.workunit.client.1.smithi155.stdout: 10 9098 9.96 MB/sec warmup 65 sec latency 140.011 ms 2024-09-19T06:31:19.841 INFO:tasks.workunit.client.0.smithi053.stdout: 10 32647 30.68 MB/sec warmup 63 sec latency 433.737 ms 2024-09-19T06:31:20.024 INFO:tasks.workunit.client.1.smithi155.stdout: 10 9216 9.83 MB/sec warmup 66 sec latency 170.104 ms 2024-09-19T06:31:20.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:20 smithi155 ceph-mon[35292]: pgmap v655: 129 pgs: 129 active+clean; 392 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 9.5 KiB/s rd, 3.0 MiB/s wr, 183 op/s 2024-09-19T06:31:20.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:20 smithi053 ceph-mon[27968]: pgmap v655: 129 pgs: 129 active+clean; 392 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 9.5 KiB/s rd, 3.0 MiB/s wr, 183 op/s 2024-09-19T06:31:20.841 INFO:tasks.workunit.client.0.smithi053.stdout: 10 32662 30.21 MB/sec warmup 64 sec latency 458.886 ms 2024-09-19T06:31:21.024 INFO:tasks.workunit.client.1.smithi155.stdout: 10 9343 9.76 MB/sec warmup 67 sec latency 190.589 ms 2024-09-19T06:31:21.841 INFO:tasks.workunit.client.0.smithi053.stdout: 10 32699 29.76 MB/sec warmup 65 sec latency 459.378 ms 2024-09-19T06:31:22.024 INFO:tasks.workunit.client.1.smithi155.stdout: 10 9453 9.72 MB/sec warmup 68 sec latency 175.542 ms 2024-09-19T06:31:22.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:22 smithi155 ceph-mon[35292]: pgmap v656: 129 pgs: 129 active+clean; 392 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 6.8 KiB/s rd, 2.4 MiB/s wr, 182 op/s 2024-09-19T06:31:22.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:22 smithi053 ceph-mon[27968]: pgmap v656: 129 pgs: 129 active+clean; 392 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 6.8 KiB/s rd, 2.4 MiB/s wr, 182 op/s 2024-09-19T06:31:22.841 INFO:tasks.workunit.client.0.smithi053.stdout: 10 32712 29.31 MB/sec warmup 66 sec latency 412.836 ms 2024-09-19T06:31:23.024 INFO:tasks.workunit.client.1.smithi155.stdout: 10 9547 9.64 MB/sec warmup 69 sec latency 167.906 ms 2024-09-19T06:31:23.841 INFO:tasks.workunit.client.0.smithi053.stdout: 10 32766 28.89 MB/sec warmup 67 sec latency 450.415 ms 2024-09-19T06:31:24.024 INFO:tasks.workunit.client.1.smithi155.stdout: 10 9640 9.58 MB/sec warmup 70 sec latency 186.268 ms 2024-09-19T06:31:24.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:24 smithi155 ceph-mon[35292]: pgmap v657: 129 pgs: 129 active+clean; 395 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 7.9 KiB/s rd, 2.9 MiB/s wr, 221 op/s 2024-09-19T06:31:24.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:24 smithi053 ceph-mon[27968]: pgmap v657: 129 pgs: 129 active+clean; 395 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 7.9 KiB/s rd, 2.9 MiB/s wr, 221 op/s 2024-09-19T06:31:24.841 INFO:tasks.workunit.client.0.smithi053.stdout: 10 32771 28.47 MB/sec warmup 68 sec latency 456.366 ms 2024-09-19T06:31:25.024 INFO:tasks.workunit.client.1.smithi155.stdout: 10 9766 9.64 MB/sec warmup 71 sec latency 181.902 ms 2024-09-19T06:31:25.841 INFO:tasks.workunit.client.0.smithi053.stdout: 10 32791 28.06 MB/sec warmup 69 sec latency 442.173 ms 2024-09-19T06:31:26.024 INFO:tasks.workunit.client.1.smithi155.stdout: 10 9944 9.84 MB/sec warmup 72 sec latency 236.186 ms 2024-09-19T06:31:26.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:26 smithi155 ceph-mon[35292]: pgmap v658: 129 pgs: 129 active+clean; 397 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.4 MiB/s wr, 148 op/s 2024-09-19T06:31:26.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:26 smithi053 ceph-mon[27968]: pgmap v658: 129 pgs: 129 active+clean; 397 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.4 MiB/s wr, 148 op/s 2024-09-19T06:31:26.841 INFO:tasks.workunit.client.0.smithi053.stdout: 10 32858 27.69 MB/sec warmup 70 sec latency 501.235 ms 2024-09-19T06:31:27.024 INFO:tasks.workunit.client.1.smithi155.stdout: 10 10181 9.90 MB/sec warmup 73 sec latency 212.192 ms 2024-09-19T06:31:27.841 INFO:tasks.workunit.client.0.smithi053.stdout: 10 32904 27.31 MB/sec warmup 71 sec latency 493.944 ms 2024-09-19T06:31:28.024 INFO:tasks.workunit.client.1.smithi155.stdout: 10 10381 9.94 MB/sec warmup 74 sec latency 173.409 ms 2024-09-19T06:31:28.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:27 smithi155 ceph-mon[35292]: pgmap v659: 129 pgs: 129 active+clean; 401 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.5 MiB/s wr, 136 op/s 2024-09-19T06:31:28.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:27 smithi053 ceph-mon[27968]: pgmap v659: 129 pgs: 129 active+clean; 401 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.5 MiB/s wr, 136 op/s 2024-09-19T06:31:28.842 INFO:tasks.workunit.client.0.smithi053.stdout: 10 33135 27.13 MB/sec warmup 72 sec latency 573.961 ms 2024-09-19T06:31:29.024 INFO:tasks.workunit.client.1.smithi155.stdout: 10 10407 9.81 MB/sec warmup 75 sec latency 145.639 ms 2024-09-19T06:31:29.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:28 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:31:29.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:28 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:31:29.842 INFO:tasks.workunit.client.0.smithi053.stdout: 10 34321 27.40 MB/sec warmup 73 sec latency 461.063 ms 2024-09-19T06:31:30.024 INFO:tasks.workunit.client.1.smithi155.stdout: 10 10431 9.69 MB/sec warmup 76 sec latency 138.656 ms 2024-09-19T06:31:30.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:29 smithi155 ceph-mon[35292]: pgmap v660: 129 pgs: 129 active+clean; 413 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 4.4 KiB/s rd, 3.4 MiB/s wr, 152 op/s 2024-09-19T06:31:30.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:29 smithi053 ceph-mon[27968]: pgmap v660: 129 pgs: 129 active+clean; 413 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 4.4 KiB/s rd, 3.4 MiB/s wr, 152 op/s 2024-09-19T06:31:30.842 INFO:tasks.workunit.client.0.smithi053.stdout: 10 35382 28.11 MB/sec warmup 74 sec latency 176.550 ms 2024-09-19T06:31:31.024 INFO:tasks.workunit.client.1.smithi155.stdout: 10 10545 9.76 MB/sec warmup 77 sec latency 112.922 ms 2024-09-19T06:31:31.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:30 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:31:31.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:30 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:31:31.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:30 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:31:31.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:30 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:31:31.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:30 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:31:31.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:30 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:31:31.842 INFO:tasks.workunit.client.0.smithi053.stdout: 10 36262 28.35 MB/sec warmup 75 sec latency 129.167 ms 2024-09-19T06:31:32.024 INFO:tasks.workunit.client.1.smithi155.stdout: 10 10744 9.88 MB/sec warmup 78 sec latency 107.975 ms 2024-09-19T06:31:32.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:31 smithi155 ceph-mon[35292]: pgmap v661: 129 pgs: 129 active+clean; 415 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 3.0 MiB/s wr, 97 op/s 2024-09-19T06:31:32.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:31 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:31:32.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:31 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:31:32.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:31 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:31:32.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:31 smithi053 ceph-mon[27968]: pgmap v661: 129 pgs: 129 active+clean; 415 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 3.0 MiB/s wr, 97 op/s 2024-09-19T06:31:32.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:31 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:31:32.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:31 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:31:32.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:31 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:31:32.842 INFO:tasks.workunit.client.0.smithi053.stdout: 10 37141 28.61 MB/sec warmup 76 sec latency 92.706 ms 2024-09-19T06:31:33.024 INFO:tasks.workunit.client.1.smithi155.stdout: 10 10899 9.87 MB/sec warmup 79 sec latency 110.596 ms 2024-09-19T06:31:33.842 INFO:tasks.workunit.client.0.smithi053.stdout: 10 38409 29.15 MB/sec warmup 77 sec latency 117.900 ms 2024-09-19T06:31:34.024 INFO:tasks.workunit.client.1.smithi155.stdout: 10 11002 9.77 MB/sec warmup 80 sec latency 103.273 ms 2024-09-19T06:31:34.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:34 smithi053 ceph-mon[27968]: pgmap v662: 129 pgs: 129 active+clean; 421 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 4.2 KiB/s rd, 4.2 MiB/s wr, 126 op/s 2024-09-19T06:31:34.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:34 smithi155 ceph-mon[35292]: pgmap v662: 129 pgs: 129 active+clean; 421 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 4.2 KiB/s rd, 4.2 MiB/s wr, 126 op/s 2024-09-19T06:31:34.842 INFO:tasks.workunit.client.0.smithi053.stdout: 10 39021 29.18 MB/sec warmup 78 sec latency 365.856 ms 2024-09-19T06:31:35.025 INFO:tasks.workunit.client.1.smithi155.stdout: 10 11089 9.67 MB/sec warmup 81 sec latency 173.069 ms 2024-09-19T06:31:35.842 INFO:tasks.workunit.client.0.smithi053.stdout: 10 39736 29.31 MB/sec warmup 79 sec latency 547.297 ms 2024-09-19T06:31:36.024 INFO:tasks.workunit.client.1.smithi155.stdout: 10 11162 9.55 MB/sec warmup 82 sec latency 218.677 ms 2024-09-19T06:31:36.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:36 smithi053 ceph-mon[27968]: pgmap v663: 129 pgs: 129 active+clean; 422 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 3.1 KiB/s rd, 4.1 MiB/s wr, 91 op/s 2024-09-19T06:31:36.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:36 smithi155 ceph-mon[35292]: pgmap v663: 129 pgs: 129 active+clean; 422 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 3.1 KiB/s rd, 4.1 MiB/s wr, 91 op/s 2024-09-19T06:31:36.842 INFO:tasks.workunit.client.0.smithi053.stdout: 10 39745 28.94 MB/sec warmup 80 sec latency 425.519 ms 2024-09-19T06:31:37.024 INFO:tasks.workunit.client.1.smithi155.stdout: 10 11246 9.44 MB/sec warmup 83 sec latency 202.603 ms 2024-09-19T06:31:37.842 INFO:tasks.workunit.client.0.smithi053.stdout: 10 39784 28.60 MB/sec warmup 81 sec latency 475.719 ms 2024-09-19T06:31:38.024 INFO:tasks.workunit.client.1.smithi155.stdout: 10 11333 9.34 MB/sec warmup 84 sec latency 203.998 ms 2024-09-19T06:31:38.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:38 smithi053 ceph-mon[27968]: pgmap v664: 129 pgs: 129 active+clean; 423 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 3.1 KiB/s rd, 4.1 MiB/s wr, 88 op/s 2024-09-19T06:31:38.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:38 smithi155 ceph-mon[35292]: pgmap v664: 129 pgs: 129 active+clean; 423 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 3.1 KiB/s rd, 4.1 MiB/s wr, 88 op/s 2024-09-19T06:31:38.842 INFO:tasks.workunit.client.0.smithi053.stdout: 10 39795 28.25 MB/sec warmup 82 sec latency 443.141 ms 2024-09-19T06:31:39.024 INFO:tasks.workunit.client.1.smithi155.stdout: 10 11428 9.26 MB/sec warmup 85 sec latency 215.378 ms 2024-09-19T06:31:39.842 INFO:tasks.workunit.client.0.smithi053.stdout: 10 39874 27.94 MB/sec warmup 83 sec latency 413.426 ms 2024-09-19T06:31:40.024 INFO:tasks.workunit.client.1.smithi155.stdout: 10 11554 9.19 MB/sec warmup 86 sec latency 168.179 ms 2024-09-19T06:31:40.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:40 smithi053 ceph-mon[27968]: pgmap v665: 129 pgs: 129 active+clean; 426 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 6.6 KiB/s rd, 4.0 MiB/s wr, 107 op/s 2024-09-19T06:31:40.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:40 smithi155 ceph-mon[35292]: pgmap v665: 129 pgs: 129 active+clean; 426 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 6.6 KiB/s rd, 4.0 MiB/s wr, 107 op/s 2024-09-19T06:31:40.842 INFO:tasks.workunit.client.0.smithi053.stdout: 10 40994 28.20 MB/sec warmup 84 sec latency 109.936 ms 2024-09-19T06:31:41.024 INFO:tasks.workunit.client.1.smithi155.stdout: 10 11705 9.19 MB/sec warmup 87 sec latency 94.299 ms 2024-09-19T06:31:41.842 INFO:tasks.workunit.client.0.smithi053.stdout: 10 42271 28.81 MB/sec warmup 85 sec latency 113.883 ms 2024-09-19T06:31:42.024 INFO:tasks.workunit.client.1.smithi155.stdout: 10 11849 9.14 MB/sec warmup 88 sec latency 106.540 ms 2024-09-19T06:31:42.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:42 smithi053 ceph-mon[27968]: pgmap v666: 129 pgs: 129 active+clean; 424 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.9 MiB/s wr, 83 op/s 2024-09-19T06:31:42.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:42 smithi155 ceph-mon[35292]: pgmap v666: 129 pgs: 129 active+clean; 424 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.9 MiB/s wr, 83 op/s 2024-09-19T06:31:42.842 INFO:tasks.workunit.client.0.smithi053.stdout: 10 43290 29.15 MB/sec warmup 86 sec latency 209.071 ms 2024-09-19T06:31:43.025 INFO:tasks.workunit.client.1.smithi155.stdout: 10 12035 9.27 MB/sec warmup 89 sec latency 101.350 ms 2024-09-19T06:31:43.842 INFO:tasks.workunit.client.0.smithi053.stdout: 10 43520 28.88 MB/sec warmup 87 sec latency 92.146 ms 2024-09-19T06:31:44.025 INFO:tasks.workunit.client.1.smithi155.stdout: 10 12213 9.27 MB/sec warmup 90 sec latency 83.392 ms 2024-09-19T06:31:44.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:44 smithi053 ceph-mon[27968]: pgmap v667: 129 pgs: 129 active+clean; 408 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 8.9 KiB/s rd, 3.9 MiB/s wr, 168 op/s 2024-09-19T06:31:44.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:44 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:31:44.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:44 smithi155 ceph-mon[35292]: pgmap v667: 129 pgs: 129 active+clean; 408 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 8.9 KiB/s rd, 3.9 MiB/s wr, 168 op/s 2024-09-19T06:31:44.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:44 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:31:44.844 INFO:tasks.workunit.client.0.smithi053.stdout: 10 44860 29.22 MB/sec warmup 88 sec latency 188.457 ms 2024-09-19T06:31:45.025 INFO:tasks.workunit.client.1.smithi155.stdout: 10 12369 9.19 MB/sec warmup 91 sec latency 145.149 ms 2024-09-19T06:31:45.842 INFO:tasks.workunit.client.0.smithi053.stdout: 10 45056 28.99 MB/sec warmup 89 sec latency 374.064 ms 2024-09-19T06:31:46.025 INFO:tasks.workunit.client.1.smithi155.stdout: 10 12525 9.10 MB/sec warmup 92 sec latency 149.665 ms 2024-09-19T06:31:46.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:46 smithi053 ceph-mon[27968]: pgmap v668: 129 pgs: 129 active+clean; 406 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 7.5 KiB/s rd, 3.0 MiB/s wr, 145 op/s 2024-09-19T06:31:46.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:46 smithi155 ceph-mon[35292]: pgmap v668: 129 pgs: 129 active+clean; 406 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 7.5 KiB/s rd, 3.0 MiB/s wr, 145 op/s 2024-09-19T06:31:46.842 INFO:tasks.workunit.client.0.smithi053.stdout: 10 45402 29.08 MB/sec warmup 90 sec latency 490.446 ms 2024-09-19T06:31:47.025 INFO:tasks.workunit.client.1.smithi155.stdout: 10 12665 9.02 MB/sec warmup 93 sec latency 167.809 ms 2024-09-19T06:31:47.842 INFO:tasks.workunit.client.0.smithi053.stdout: 10 45612 28.92 MB/sec warmup 91 sec latency 514.529 ms 2024-09-19T06:31:48.025 INFO:tasks.workunit.client.1.smithi155.stdout: 10 12784 8.94 MB/sec warmup 94 sec latency 190.277 ms 2024-09-19T06:31:48.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:48 smithi053 ceph-mon[27968]: pgmap v669: 129 pgs: 129 active+clean; 406 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 7.5 KiB/s rd, 2.5 MiB/s wr, 151 op/s 2024-09-19T06:31:48.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:48 smithi155 ceph-mon[35292]: pgmap v669: 129 pgs: 129 active+clean; 406 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 7.5 KiB/s rd, 2.5 MiB/s wr, 151 op/s 2024-09-19T06:31:48.842 INFO:tasks.workunit.client.0.smithi053.stdout: 10 45898 28.79 MB/sec warmup 92 sec latency 523.770 ms 2024-09-19T06:31:49.025 INFO:tasks.workunit.client.1.smithi155.stdout: 10 12940 8.93 MB/sec warmup 95 sec latency 166.767 ms 2024-09-19T06:31:49.843 INFO:tasks.workunit.client.0.smithi053.stdout: 10 46022 28.53 MB/sec warmup 93 sec latency 394.063 ms 2024-09-19T06:31:50.025 INFO:tasks.workunit.client.1.smithi155.stdout: 10 13041 8.90 MB/sec warmup 96 sec latency 196.164 ms 2024-09-19T06:31:50.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:50 smithi053 ceph-mon[27968]: pgmap v670: 129 pgs: 129 active+clean; 408 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 9.3 KiB/s rd, 2.4 MiB/s wr, 197 op/s 2024-09-19T06:31:50.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:50 smithi155 ceph-mon[35292]: pgmap v670: 129 pgs: 129 active+clean; 408 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 9.3 KiB/s rd, 2.4 MiB/s wr, 197 op/s 2024-09-19T06:31:50.843 INFO:tasks.workunit.client.0.smithi053.stdout: 10 46896 28.72 MB/sec warmup 94 sec latency 436.139 ms 2024-09-19T06:31:51.025 INFO:tasks.workunit.client.1.smithi155.stdout: 10 13166 8.87 MB/sec warmup 97 sec latency 89.781 ms 2024-09-19T06:31:51.843 INFO:tasks.workunit.client.0.smithi053.stdout: 10 47690 28.92 MB/sec warmup 95 sec latency 96.131 ms 2024-09-19T06:31:52.025 INFO:tasks.workunit.client.1.smithi155.stdout: 10 13389 9.03 MB/sec warmup 98 sec latency 70.402 ms 2024-09-19T06:31:52.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:52 smithi053 ceph-mon[27968]: pgmap v671: 129 pgs: 129 active+clean; 408 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 5.8 KiB/s rd, 2.0 MiB/s wr, 179 op/s 2024-09-19T06:31:52.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:52 smithi155 ceph-mon[35292]: pgmap v671: 129 pgs: 129 active+clean; 408 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 5.8 KiB/s rd, 2.0 MiB/s wr, 179 op/s 2024-09-19T06:31:52.843 INFO:tasks.workunit.client.0.smithi053.stdout: 10 48890 29.15 MB/sec warmup 96 sec latency 175.918 ms 2024-09-19T06:31:53.025 INFO:tasks.workunit.client.1.smithi155.stdout: 10 13698 9.22 MB/sec warmup 99 sec latency 84.402 ms 2024-09-19T06:31:53.843 INFO:tasks.workunit.client.0.smithi053.stdout: 10 49932 29.66 MB/sec warmup 97 sec latency 247.262 ms 2024-09-19T06:31:54.025 INFO:tasks.workunit.client.1.smithi155.stdout: 10 13932 9.27 MB/sec warmup 100 sec latency 106.038 ms 2024-09-19T06:31:54.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:54 smithi053 ceph-mon[27968]: pgmap v672: 129 pgs: 129 active+clean; 401 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 3.6 MiB/s wr, 221 op/s 2024-09-19T06:31:54.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:54 smithi155 ceph-mon[35292]: pgmap v672: 129 pgs: 129 active+clean; 401 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 3.6 MiB/s wr, 221 op/s 2024-09-19T06:31:54.843 INFO:tasks.workunit.client.0.smithi053.stdout: 10 50490 29.51 MB/sec warmup 98 sec latency 93.981 ms 2024-09-19T06:31:55.025 INFO:tasks.workunit.client.1.smithi155.stdout: 10 13960 9.18 MB/sec warmup 101 sec latency 107.612 ms 2024-09-19T06:31:55.843 INFO:tasks.workunit.client.0.smithi053.stdout: 10 51353 29.69 MB/sec warmup 99 sec latency 102.839 ms 2024-09-19T06:31:56.025 INFO:tasks.workunit.client.1.smithi155.stdout: 10 13992 9.09 MB/sec warmup 102 sec latency 89.270 ms 2024-09-19T06:31:56.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:56 smithi053 ceph-mon[27968]: pgmap v673: 129 pgs: 129 active+clean; 404 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 3.2 MiB/s wr, 142 op/s 2024-09-19T06:31:56.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:56 smithi155 ceph-mon[35292]: pgmap v673: 129 pgs: 129 active+clean; 404 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 3.2 MiB/s wr, 142 op/s 2024-09-19T06:31:56.843 INFO:tasks.workunit.client.0.smithi053.stdout: 10 52642 30.13 MB/sec warmup 100 sec latency 166.687 ms 2024-09-19T06:31:57.025 INFO:tasks.workunit.client.1.smithi155.stdout: 10 14168 9.22 MB/sec warmup 103 sec latency 86.990 ms 2024-09-19T06:31:57.843 INFO:tasks.workunit.client.0.smithi053.stdout: 10 53918 30.50 MB/sec warmup 101 sec latency 126.809 ms 2024-09-19T06:31:58.025 INFO:tasks.workunit.client.1.smithi155.stdout: 10 14356 9.27 MB/sec warmup 104 sec latency 113.845 ms 2024-09-19T06:31:58.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:58 smithi053 ceph-mon[27968]: pgmap v674: 129 pgs: 129 active+clean; 406 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 3.4 MiB/s wr, 134 op/s 2024-09-19T06:31:58.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:58 smithi155 ceph-mon[35292]: pgmap v674: 129 pgs: 129 active+clean; 406 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 3.4 MiB/s wr, 134 op/s 2024-09-19T06:31:58.843 INFO:tasks.workunit.client.0.smithi053.stdout: 10 54038 30.23 MB/sec warmup 102 sec latency 116.400 ms 2024-09-19T06:31:59.025 INFO:tasks.workunit.client.1.smithi155.stdout: 10 14484 9.26 MB/sec warmup 105 sec latency 138.575 ms 2024-09-19T06:31:59.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:31:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:31:59.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:31:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:31:59.843 INFO:tasks.workunit.client.0.smithi053.stdout: 10 54074 29.94 MB/sec warmup 103 sec latency 437.866 ms 2024-09-19T06:32:00.025 INFO:tasks.workunit.client.1.smithi155.stdout: 10 14585 9.19 MB/sec warmup 106 sec latency 115.791 ms 2024-09-19T06:32:00.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:00 smithi053 ceph-mon[27968]: pgmap v675: 129 pgs: 129 active+clean; 411 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 4.8 KiB/s rd, 4.3 MiB/s wr, 161 op/s 2024-09-19T06:32:00.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:00 smithi155 ceph-mon[35292]: pgmap v675: 129 pgs: 129 active+clean; 411 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 4.8 KiB/s rd, 4.3 MiB/s wr, 161 op/s 2024-09-19T06:32:00.843 INFO:tasks.workunit.client.0.smithi053.stdout: 10 54884 30.10 MB/sec warmup 104 sec latency 389.510 ms 2024-09-19T06:32:01.025 INFO:tasks.workunit.client.1.smithi155.stdout: 10 14690 9.11 MB/sec warmup 107 sec latency 92.756 ms 2024-09-19T06:32:01.843 INFO:tasks.workunit.client.0.smithi053.stdout: 10 56155 30.49 MB/sec warmup 105 sec latency 126.047 ms 2024-09-19T06:32:02.025 INFO:tasks.workunit.client.1.smithi155.stdout: 10 14807 9.03 MB/sec warmup 108 sec latency 171.952 ms 2024-09-19T06:32:02.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:02 smithi053 ceph-mon[27968]: pgmap v676: 129 pgs: 129 active+clean; 412 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 4.4 MiB/s wr, 113 op/s 2024-09-19T06:32:02.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:02 smithi155 ceph-mon[35292]: pgmap v676: 129 pgs: 129 active+clean; 412 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 4.4 MiB/s wr, 113 op/s 2024-09-19T06:32:02.843 INFO:tasks.workunit.client.0.smithi053.stdout: 10 57462 30.86 MB/sec warmup 106 sec latency 125.025 ms 2024-09-19T06:32:03.025 INFO:tasks.workunit.client.1.smithi155.stdout: 10 14940 8.97 MB/sec warmup 109 sec latency 153.255 ms 2024-09-19T06:32:03.843 INFO:tasks.workunit.client.0.smithi053.stdout: 10 57576 30.60 MB/sec warmup 107 sec latency 167.335 ms 2024-09-19T06:32:04.025 INFO:tasks.workunit.client.1.smithi155.stdout: 10 15087 8.92 MB/sec warmup 110 sec latency 118.573 ms 2024-09-19T06:32:04.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:04 smithi053 ceph-mon[27968]: pgmap v677: 129 pgs: 129 active+clean; 417 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 6.7 KiB/s rd, 5.3 MiB/s wr, 141 op/s 2024-09-19T06:32:04.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:04 smithi155 ceph-mon[35292]: pgmap v677: 129 pgs: 129 active+clean; 417 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 6.7 KiB/s rd, 5.3 MiB/s wr, 141 op/s 2024-09-19T06:32:04.843 INFO:tasks.workunit.client.0.smithi053.stdout: 10 58323 30.75 MB/sec warmup 108 sec latency 117.338 ms 2024-09-19T06:32:05.025 INFO:tasks.workunit.client.1.smithi155.stdout: 10 15233 8.92 MB/sec warmup 111 sec latency 122.906 ms 2024-09-19T06:32:05.843 INFO:tasks.workunit.client.0.smithi053.stdout: 10 58767 30.52 MB/sec warmup 109 sec latency 473.064 ms 2024-09-19T06:32:06.025 INFO:tasks.workunit.client.1.smithi155.stdout: 10 15379 8.88 MB/sec warmup 112 sec latency 158.307 ms 2024-09-19T06:32:06.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:06 smithi053 ceph-mon[27968]: pgmap v678: 129 pgs: 129 active+clean; 414 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 5.4 KiB/s rd, 3.9 MiB/s wr, 105 op/s 2024-09-19T06:32:06.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:06 smithi155 ceph-mon[35292]: pgmap v678: 129 pgs: 129 active+clean; 414 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 5.4 KiB/s rd, 3.9 MiB/s wr, 105 op/s 2024-09-19T06:32:06.843 INFO:tasks.workunit.client.0.smithi053.stdout: 10 59775 30.87 MB/sec warmup 110 sec latency 368.258 ms 2024-09-19T06:32:07.025 INFO:tasks.workunit.client.1.smithi155.stdout: 10 15547 8.94 MB/sec warmup 113 sec latency 129.709 ms 2024-09-19T06:32:07.844 INFO:tasks.workunit.client.0.smithi053.stdout: 10 61007 31.19 MB/sec warmup 111 sec latency 143.058 ms 2024-09-19T06:32:08.025 INFO:tasks.workunit.client.1.smithi155.stdout: 10 15727 9.00 MB/sec warmup 114 sec latency 70.639 ms 2024-09-19T06:32:08.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:08 smithi053 ceph-mon[27968]: pgmap v679: 129 pgs: 129 active+clean; 408 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 5.4 KiB/s rd, 3.5 MiB/s wr, 116 op/s 2024-09-19T06:32:08.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:08 smithi155 ceph-mon[35292]: pgmap v679: 129 pgs: 129 active+clean; 408 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 5.4 KiB/s rd, 3.5 MiB/s wr, 116 op/s 2024-09-19T06:32:08.844 INFO:tasks.workunit.client.0.smithi053.stdout: 10 61084 30.93 MB/sec warmup 112 sec latency 271.024 ms 2024-09-19T06:32:09.025 INFO:tasks.workunit.client.1.smithi155.stdout: 10 15884 8.93 MB/sec warmup 115 sec latency 146.478 ms 2024-09-19T06:32:09.844 INFO:tasks.workunit.client.0.smithi053.stdout: 10 61797 31.04 MB/sec warmup 113 sec latency 102.573 ms 2024-09-19T06:32:10.026 INFO:tasks.workunit.client.1.smithi155.stdout: 10 16082 8.87 MB/sec warmup 116 sec latency 74.823 ms 2024-09-19T06:32:10.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:10 smithi053 ceph-mon[27968]: pgmap v680: 129 pgs: 129 active+clean; 398 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 9.9 KiB/s rd, 3.9 MiB/s wr, 190 op/s 2024-09-19T06:32:10.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:10 smithi155 ceph-mon[35292]: pgmap v680: 129 pgs: 129 active+clean; 398 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 9.9 KiB/s rd, 3.9 MiB/s wr, 190 op/s 2024-09-19T06:32:10.844 INFO:tasks.workunit.client.0.smithi053.stdout: 10 63246 31.44 MB/sec warmup 114 sec latency 109.108 ms 2024-09-19T06:32:11.026 INFO:tasks.workunit.client.1.smithi155.stdout: 10 16258 8.81 MB/sec warmup 117 sec latency 86.627 ms 2024-09-19T06:32:11.844 INFO:tasks.workunit.client.0.smithi053.stdout: 10 63744 31.39 MB/sec warmup 115 sec latency 324.874 ms 2024-09-19T06:32:12.026 INFO:tasks.workunit.client.1.smithi155.stdout: 10 16406 8.77 MB/sec warmup 118 sec latency 135.835 ms 2024-09-19T06:32:12.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:12 smithi053 ceph-mon[27968]: pgmap v681: 129 pgs: 129 active+clean; 397 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 8.2 KiB/s rd, 3.1 MiB/s wr, 158 op/s 2024-09-19T06:32:12.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:12 smithi155 ceph-mon[35292]: pgmap v681: 129 pgs: 129 active+clean; 397 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 8.2 KiB/s rd, 3.1 MiB/s wr, 158 op/s 2024-09-19T06:32:12.844 INFO:tasks.workunit.client.0.smithi053.stdout: 10 64551 31.50 MB/sec warmup 116 sec latency 339.152 ms 2024-09-19T06:32:13.026 INFO:tasks.workunit.client.1.smithi155.stdout: 10 16543 8.77 MB/sec warmup 119 sec latency 129.045 ms 2024-09-19T06:32:13.844 INFO:tasks.workunit.client.0.smithi053.stdout: 10 64602 31.24 MB/sec warmup 117 sec latency 317.310 ms 2024-09-19T06:32:14.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:14 smithi053 ceph-mon[27968]: pgmap v682: 129 pgs: 129 active+clean; 400 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 3.2 MiB/s wr, 219 op/s 2024-09-19T06:32:14.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:14 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:32:14.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:14 smithi155 ceph-mon[35292]: pgmap v682: 129 pgs: 129 active+clean; 400 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 3.2 MiB/s wr, 219 op/s 2024-09-19T06:32:14.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:14 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:32:14.844 INFO:tasks.workunit.client.0.smithi053.stdout: 10 64610 30.98 MB/sec warmup 118 sec latency 397.474 ms 2024-09-19T06:32:15.026 INFO:tasks.workunit.client.1.smithi155.stdout: 10 16748 8.84 MB/sec execute 1 sec latency 159.827 ms 2024-09-19T06:32:15.844 INFO:tasks.workunit.client.0.smithi053.stdout: 10 64672 30.73 MB/sec warmup 119 sec latency 267.132 ms 2024-09-19T06:32:16.026 INFO:tasks.workunit.client.1.smithi155.stdout: 10 16930 12.95 MB/sec execute 2 sec latency 134.742 ms 2024-09-19T06:32:16.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:16 smithi053 ceph-mon[27968]: pgmap v683: 129 pgs: 129 active+clean; 401 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 6.2 KiB/s rd, 2.6 MiB/s wr, 193 op/s 2024-09-19T06:32:16.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:16 smithi155 ceph-mon[35292]: pgmap v683: 129 pgs: 129 active+clean; 401 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 6.2 KiB/s rd, 2.6 MiB/s wr, 193 op/s 2024-09-19T06:32:17.026 INFO:tasks.workunit.client.1.smithi155.stdout: 10 17167 17.02 MB/sec execute 3 sec latency 138.192 ms 2024-09-19T06:32:17.844 INFO:tasks.workunit.client.0.smithi053.stdout: 10 64801 2.85 MB/sec execute 1 sec latency 333.874 ms 2024-09-19T06:32:18.028 INFO:tasks.workunit.client.1.smithi155.stdout: 10 17373 15.65 MB/sec execute 4 sec latency 242.036 ms 2024-09-19T06:32:18.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:18 smithi053 ceph-mon[27968]: pgmap v684: 129 pgs: 129 active+clean; 403 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 6.2 KiB/s rd, 2.9 MiB/s wr, 190 op/s 2024-09-19T06:32:18.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:18 smithi155 ceph-mon[35292]: pgmap v684: 129 pgs: 129 active+clean; 403 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 6.2 KiB/s rd, 2.9 MiB/s wr, 190 op/s 2024-09-19T06:32:18.844 INFO:tasks.workunit.client.0.smithi053.stdout: 10 65387 22.86 MB/sec execute 2 sec latency 378.899 ms 2024-09-19T06:32:19.026 INFO:tasks.workunit.client.1.smithi155.stdout: 10 17485 14.21 MB/sec execute 5 sec latency 164.590 ms 2024-09-19T06:32:19.844 INFO:tasks.workunit.client.0.smithi053.stdout: 10 65807 16.77 MB/sec execute 3 sec latency 319.379 ms 2024-09-19T06:32:20.026 INFO:tasks.workunit.client.1.smithi155.stdout: 10 17506 11.86 MB/sec execute 6 sec latency 122.616 ms 2024-09-19T06:32:20.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:20 smithi053 ceph-mon[27968]: pgmap v685: 129 pgs: 129 active+clean; 418 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 7.9 KiB/s rd, 4.0 MiB/s wr, 206 op/s 2024-09-19T06:32:20.485 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:20 smithi155 ceph-mon[35292]: pgmap v685: 129 pgs: 129 active+clean; 418 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 7.9 KiB/s rd, 4.0 MiB/s wr, 206 op/s 2024-09-19T06:32:20.844 INFO:tasks.workunit.client.0.smithi053.stdout: 10 66278 16.52 MB/sec execute 4 sec latency 335.265 ms 2024-09-19T06:32:21.026 INFO:tasks.workunit.client.1.smithi155.stdout: 10 17533 10.21 MB/sec execute 7 sec latency 145.450 ms 2024-09-19T06:32:21.844 INFO:tasks.workunit.client.0.smithi053.stdout: 10 66803 23.99 MB/sec execute 5 sec latency 338.553 ms 2024-09-19T06:32:22.026 INFO:tasks.workunit.client.1.smithi155.stdout: 10 17673 10.93 MB/sec execute 8 sec latency 116.906 ms 2024-09-19T06:32:22.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:22 smithi155 ceph-mon[35292]: pgmap v686: 129 pgs: 129 active+clean; 420 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 3.4 KiB/s rd, 3.2 MiB/s wr, 128 op/s 2024-09-19T06:32:22.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:22 smithi053 ceph-mon[27968]: pgmap v686: 129 pgs: 129 active+clean; 420 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 3.4 KiB/s rd, 3.2 MiB/s wr, 128 op/s 2024-09-19T06:32:22.844 INFO:tasks.workunit.client.0.smithi053.stdout: 10 67166 23.10 MB/sec execute 6 sec latency 380.155 ms 2024-09-19T06:32:23.026 INFO:tasks.workunit.client.1.smithi155.stdout: 10 17853 11.88 MB/sec execute 9 sec latency 213.126 ms 2024-09-19T06:32:23.844 INFO:tasks.workunit.client.0.smithi053.stdout: 10 67385 21.24 MB/sec execute 7 sec latency 387.743 ms 2024-09-19T06:32:24.026 INFO:tasks.workunit.client.1.smithi155.stdout: 10 17988 11.38 MB/sec execute 10 sec latency 129.146 ms 2024-09-19T06:32:24.135 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:24 smithi053 ceph-mon[27968]: pgmap v687: 129 pgs: 129 active+clean; 426 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 4.7 KiB/s rd, 4.0 MiB/s wr, 154 op/s 2024-09-19T06:32:24.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:24 smithi155 ceph-mon[35292]: pgmap v687: 129 pgs: 129 active+clean; 426 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 4.7 KiB/s rd, 4.0 MiB/s wr, 154 op/s 2024-09-19T06:32:24.844 INFO:tasks.workunit.client.0.smithi053.stdout: 10 68101 23.50 MB/sec execute 8 sec latency 433.963 ms 2024-09-19T06:32:25.026 INFO:tasks.workunit.client.1.smithi155.stdout: 10 18056 10.55 MB/sec execute 11 sec latency 162.530 ms 2024-09-19T06:32:25.844 INFO:tasks.workunit.client.0.smithi053.stdout: 10 68106 20.90 MB/sec execute 9 sec latency 392.261 ms 2024-09-19T06:32:26.026 INFO:tasks.workunit.client.1.smithi155.stdout: 10 18118 9.80 MB/sec execute 12 sec latency 195.144 ms 2024-09-19T06:32:26.406 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:26 smithi053 ceph-mon[27968]: pgmap v688: 129 pgs: 129 active+clean; 426 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 2.9 KiB/s rd, 4.0 MiB/s wr, 95 op/s 2024-09-19T06:32:26.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:26 smithi155 ceph-mon[35292]: pgmap v688: 129 pgs: 129 active+clean; 426 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 2.9 KiB/s rd, 4.0 MiB/s wr, 95 op/s 2024-09-19T06:32:26.844 INFO:tasks.workunit.client.0.smithi053.stdout: 10 68155 18.93 MB/sec execute 10 sec latency 434.148 ms 2024-09-19T06:32:27.026 INFO:tasks.workunit.client.1.smithi155.stdout: 10 18190 9.09 MB/sec execute 13 sec latency 191.983 ms 2024-09-19T06:32:27.845 INFO:tasks.workunit.client.0.smithi053.stdout: 10 68161 17.22 MB/sec execute 11 sec latency 458.738 ms 2024-09-19T06:32:28.026 INFO:tasks.workunit.client.1.smithi155.stdout: 10 18257 8.45 MB/sec execute 14 sec latency 208.506 ms 2024-09-19T06:32:28.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:28 smithi155 ceph-mon[35292]: pgmap v689: 129 pgs: 129 active+clean; 429 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 2.9 KiB/s rd, 4.2 MiB/s wr, 98 op/s 2024-09-19T06:32:28.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:28 smithi053 ceph-mon[27968]: pgmap v689: 129 pgs: 129 active+clean; 429 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 2.9 KiB/s rd, 4.2 MiB/s wr, 98 op/s 2024-09-19T06:32:28.845 INFO:tasks.workunit.client.0.smithi053.stdout: 10 68202 15.86 MB/sec execute 12 sec latency 417.159 ms 2024-09-19T06:32:29.026 INFO:tasks.workunit.client.1.smithi155.stdout: 10 18339 7.93 MB/sec execute 15 sec latency 189.450 ms 2024-09-19T06:32:29.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:29 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:32:29.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:32:29.845 INFO:tasks.workunit.client.0.smithi053.stdout: 10 68226 14.69 MB/sec execute 13 sec latency 425.552 ms 2024-09-19T06:32:30.026 INFO:tasks.workunit.client.1.smithi155.stdout: 10 18441 7.53 MB/sec execute 16 sec latency 228.554 ms 2024-09-19T06:32:30.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:30 smithi155 ceph-mon[35292]: pgmap v690: 129 pgs: 129 active+clean; 432 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 5.9 KiB/s rd, 4.2 MiB/s wr, 106 op/s 2024-09-19T06:32:30.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:30 smithi053 ceph-mon[27968]: pgmap v690: 129 pgs: 129 active+clean; 432 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 5.9 KiB/s rd, 4.2 MiB/s wr, 106 op/s 2024-09-19T06:32:30.845 INFO:tasks.workunit.client.0.smithi053.stdout: 10 68231 13.65 MB/sec execute 14 sec latency 475.751 ms 2024-09-19T06:32:31.027 INFO:tasks.workunit.client.1.smithi155.stdout: 10 18523 7.20 MB/sec execute 17 sec latency 206.211 ms 2024-09-19T06:32:31.845 INFO:tasks.workunit.client.0.smithi053.stdout: 10 68314 12.88 MB/sec execute 15 sec latency 483.832 ms 2024-09-19T06:32:32.026 INFO:tasks.workunit.client.1.smithi155.stdout: 10 18634 6.95 MB/sec execute 18 sec latency 196.496 ms 2024-09-19T06:32:32.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:32 smithi155 ceph-mon[35292]: pgmap v691: 129 pgs: 129 active+clean; 433 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 4.2 KiB/s rd, 2.8 MiB/s wr, 72 op/s 2024-09-19T06:32:32.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:32 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:32:32.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:32 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:32:32.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:32 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:32:32.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:32 smithi053 ceph-mon[27968]: pgmap v691: 129 pgs: 129 active+clean; 433 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 4.2 KiB/s rd, 2.8 MiB/s wr, 72 op/s 2024-09-19T06:32:32.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:32 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:32:32.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:32 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:32:32.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:32 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:32:32.845 INFO:tasks.workunit.client.0.smithi053.stdout: 10 68410 12.21 MB/sec execute 16 sec latency 525.459 ms 2024-09-19T06:32:33.026 INFO:tasks.workunit.client.1.smithi155.stdout: 10 18782 7.03 MB/sec execute 19 sec latency 103.154 ms 2024-09-19T06:32:33.414 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:33 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:32:33.414 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:33 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:32:33.414 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:33 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:32:33.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:33 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:32:33.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:33 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:32:33.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:33 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:32:33.845 INFO:tasks.workunit.client.0.smithi053.stdout: 10 69821 15.10 MB/sec execute 17 sec latency 325.291 ms 2024-09-19T06:32:34.026 INFO:tasks.workunit.client.1.smithi155.stdout: 10 18942 6.96 MB/sec execute 20 sec latency 95.326 ms 2024-09-19T06:32:34.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:34 smithi053 ceph-mon[27968]: pgmap v692: 129 pgs: 129 active+clean; 420 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 9.0 KiB/s rd, 3.3 MiB/s wr, 140 op/s 2024-09-19T06:32:34.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:34 smithi155 ceph-mon[35292]: pgmap v692: 129 pgs: 129 active+clean; 420 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 9.0 KiB/s rd, 3.3 MiB/s wr, 140 op/s 2024-09-19T06:32:34.845 INFO:tasks.workunit.client.0.smithi053.stdout: 10 70931 18.89 MB/sec execute 18 sec latency 112.366 ms 2024-09-19T06:32:35.028 INFO:tasks.workunit.client.1.smithi155.stdout: 10 19119 7.49 MB/sec execute 21 sec latency 130.439 ms 2024-09-19T06:32:35.845 INFO:tasks.workunit.client.0.smithi053.stdout: 10 71718 20.05 MB/sec execute 19 sec latency 195.887 ms 2024-09-19T06:32:36.027 INFO:tasks.workunit.client.1.smithi155.stdout: 10 19281 7.71 MB/sec execute 22 sec latency 135.385 ms 2024-09-19T06:32:36.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:36 smithi053 ceph-mon[27968]: pgmap v693: 129 pgs: 129 active+clean; 413 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 7.7 KiB/s rd, 2.6 MiB/s wr, 126 op/s 2024-09-19T06:32:36.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:36 smithi155 ceph-mon[35292]: pgmap v693: 129 pgs: 129 active+clean; 413 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 7.7 KiB/s rd, 2.6 MiB/s wr, 126 op/s 2024-09-19T06:32:36.845 INFO:tasks.workunit.client.0.smithi053.stdout: 10 72481 21.41 MB/sec execute 20 sec latency 96.871 ms 2024-09-19T06:32:37.027 INFO:tasks.workunit.client.1.smithi155.stdout: 10 19458 7.44 MB/sec execute 23 sec latency 77.573 ms 2024-09-19T06:32:37.845 INFO:tasks.workunit.client.0.smithi053.stdout: 10 73384 21.34 MB/sec execute 21 sec latency 331.798 ms 2024-09-19T06:32:38.027 INFO:tasks.workunit.client.1.smithi155.stdout: 10 19621 7.19 MB/sec execute 24 sec latency 151.902 ms 2024-09-19T06:32:38.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:38 smithi053 ceph-mon[27968]: pgmap v694: 129 pgs: 129 active+clean; 410 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 7.7 KiB/s rd, 2.5 MiB/s wr, 134 op/s 2024-09-19T06:32:38.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:38 smithi155 ceph-mon[35292]: pgmap v694: 129 pgs: 129 active+clean; 410 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 7.7 KiB/s rd, 2.5 MiB/s wr, 134 op/s 2024-09-19T06:32:38.845 INFO:tasks.workunit.client.0.smithi053.stdout: 10 73647 21.58 MB/sec execute 22 sec latency 471.094 ms 2024-09-19T06:32:39.027 INFO:tasks.workunit.client.1.smithi155.stdout: 10 19765 6.96 MB/sec execute 25 sec latency 135.023 ms 2024-09-19T06:32:39.845 INFO:tasks.workunit.client.0.smithi053.stdout: 10 73962 21.88 MB/sec execute 23 sec latency 525.389 ms 2024-09-19T06:32:40.027 INFO:tasks.workunit.client.1.smithi155.stdout: 10 19913 6.80 MB/sec execute 26 sec latency 196.617 ms 2024-09-19T06:32:40.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:40 smithi053 ceph-mon[27968]: pgmap v695: 129 pgs: 129 active+clean; 413 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 9.2 KiB/s rd, 2.6 MiB/s wr, 178 op/s 2024-09-19T06:32:40.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:40 smithi155 ceph-mon[35292]: pgmap v695: 129 pgs: 129 active+clean; 413 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 9.2 KiB/s rd, 2.6 MiB/s wr, 178 op/s 2024-09-19T06:32:40.845 INFO:tasks.workunit.client.0.smithi053.stdout: 10 74215 21.38 MB/sec execute 24 sec latency 514.427 ms 2024-09-19T06:32:41.027 INFO:tasks.workunit.client.1.smithi155.stdout: 10 20061 6.79 MB/sec execute 27 sec latency 178.097 ms 2024-09-19T06:32:41.845 INFO:tasks.workunit.client.0.smithi053.stdout: 10 74323 20.88 MB/sec execute 25 sec latency 464.836 ms 2024-09-19T06:32:42.027 INFO:tasks.workunit.client.1.smithi155.stdout: 10 20165 6.76 MB/sec execute 28 sec latency 175.959 ms 2024-09-19T06:32:42.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:42 smithi053 ceph-mon[27968]: pgmap v696: 129 pgs: 129 active+clean; 413 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 6.2 KiB/s rd, 2.0 MiB/s wr, 167 op/s 2024-09-19T06:32:42.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:42 smithi155 ceph-mon[35292]: pgmap v696: 129 pgs: 129 active+clean; 413 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 6.2 KiB/s rd, 2.0 MiB/s wr, 167 op/s 2024-09-19T06:32:42.845 INFO:tasks.workunit.client.0.smithi053.stdout: 10 75220 21.96 MB/sec execute 26 sec latency 477.773 ms 2024-09-19T06:32:43.027 INFO:tasks.workunit.client.1.smithi155.stdout: 10 20253 6.66 MB/sec execute 29 sec latency 179.147 ms 2024-09-19T06:32:43.845 INFO:tasks.workunit.client.0.smithi053.stdout: 10 75226 21.15 MB/sec execute 27 sec latency 534.582 ms 2024-09-19T06:32:44.027 INFO:tasks.workunit.client.1.smithi155.stdout: 10 20376 6.90 MB/sec execute 30 sec latency 205.979 ms 2024-09-19T06:32:44.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:44 smithi053 ceph-mon[27968]: pgmap v697: 129 pgs: 129 active+clean; 416 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 7.7 KiB/s rd, 2.7 MiB/s wr, 210 op/s 2024-09-19T06:32:44.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:44 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:32:44.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:44 smithi155 ceph-mon[35292]: pgmap v697: 129 pgs: 129 active+clean; 416 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 7.7 KiB/s rd, 2.7 MiB/s wr, 210 op/s 2024-09-19T06:32:44.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:44 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:32:44.845 INFO:tasks.workunit.client.0.smithi053.stdout: 10 75239 20.40 MB/sec execute 28 sec latency 417.251 ms 2024-09-19T06:32:45.027 INFO:tasks.workunit.client.1.smithi155.stdout: 10 20502 6.95 MB/sec execute 31 sec latency 169.699 ms 2024-09-19T06:32:45.846 INFO:tasks.workunit.client.0.smithi053.stdout: 10 75278 19.73 MB/sec execute 29 sec latency 459.872 ms 2024-09-19T06:32:46.027 INFO:tasks.workunit.client.1.smithi155.stdout: 10 20678 7.38 MB/sec execute 32 sec latency 230.969 ms 2024-09-19T06:32:46.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:46 smithi053 ceph-mon[27968]: pgmap v698: 129 pgs: 129 active+clean; 415 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 2.9 KiB/s rd, 2.5 MiB/s wr, 146 op/s 2024-09-19T06:32:46.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:46 smithi155 ceph-mon[35292]: pgmap v698: 129 pgs: 129 active+clean; 415 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 2.9 KiB/s rd, 2.5 MiB/s wr, 146 op/s 2024-09-19T06:32:46.845 INFO:tasks.workunit.client.0.smithi053.stdout: 10 75284 19.08 MB/sec execute 30 sec latency 491.634 ms 2024-09-19T06:32:47.027 INFO:tasks.workunit.client.1.smithi155.stdout: 10 20861 7.57 MB/sec execute 33 sec latency 201.733 ms 2024-09-19T06:32:47.846 INFO:tasks.workunit.client.0.smithi053.stdout: 10 75316 18.49 MB/sec execute 31 sec latency 467.110 ms 2024-09-19T06:32:48.027 INFO:tasks.workunit.client.1.smithi155.stdout: 10 21000 7.56 MB/sec execute 34 sec latency 189.214 ms 2024-09-19T06:32:48.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:48 smithi053 ceph-mon[27968]: pgmap v699: 129 pgs: 129 active+clean; 421 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 2.9 KiB/s rd, 2.8 MiB/s wr, 153 op/s 2024-09-19T06:32:48.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:48 smithi155 ceph-mon[35292]: pgmap v699: 129 pgs: 129 active+clean; 421 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 2.9 KiB/s rd, 2.8 MiB/s wr, 153 op/s 2024-09-19T06:32:48.846 INFO:tasks.workunit.client.0.smithi053.stdout: 10 75349 17.94 MB/sec execute 32 sec latency 450.483 ms 2024-09-19T06:32:49.027 INFO:tasks.workunit.client.1.smithi155.stdout: 10 21072 7.51 MB/sec execute 35 sec latency 188.799 ms 2024-09-19T06:32:49.846 INFO:tasks.workunit.client.0.smithi053.stdout: 10 76061 18.79 MB/sec execute 33 sec latency 450.267 ms 2024-09-19T06:32:50.027 INFO:tasks.workunit.client.1.smithi155.stdout: 10 21099 7.30 MB/sec execute 36 sec latency 65.862 ms 2024-09-19T06:32:50.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:50 smithi053 ceph-mon[27968]: pgmap v700: 129 pgs: 129 active+clean; 413 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 4.4 KiB/s rd, 3.7 MiB/s wr, 152 op/s 2024-09-19T06:32:50.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:50 smithi155 ceph-mon[35292]: pgmap v700: 129 pgs: 129 active+clean; 413 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 4.4 KiB/s rd, 3.7 MiB/s wr, 152 op/s 2024-09-19T06:32:50.846 INFO:tasks.workunit.client.0.smithi053.stdout: 10 77341 20.06 MB/sec execute 34 sec latency 165.880 ms 2024-09-19T06:32:51.027 INFO:tasks.workunit.client.1.smithi155.stdout: 10 21186 7.36 MB/sec execute 37 sec latency 94.675 ms 2024-09-19T06:32:51.846 INFO:tasks.workunit.client.0.smithi053.stdout: 10 78051 20.65 MB/sec execute 35 sec latency 369.225 ms 2024-09-19T06:32:52.028 INFO:tasks.workunit.client.1.smithi155.stdout: 10 21389 7.76 MB/sec execute 38 sec latency 101.549 ms 2024-09-19T06:32:52.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:52 smithi053 ceph-mon[27968]: pgmap v701: 129 pgs: 129 active+clean; 414 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 2.9 KiB/s rd, 3.1 MiB/s wr, 95 op/s 2024-09-19T06:32:52.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:52 smithi155 ceph-mon[35292]: pgmap v701: 129 pgs: 129 active+clean; 414 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 2.9 KiB/s rd, 3.1 MiB/s wr, 95 op/s 2024-09-19T06:32:52.846 INFO:tasks.workunit.client.0.smithi053.stdout: 10 78764 21.17 MB/sec execute 36 sec latency 501.557 ms 2024-09-19T06:32:53.027 INFO:tasks.workunit.client.1.smithi155.stdout: 10 21514 7.71 MB/sec execute 39 sec latency 181.386 ms 2024-09-19T06:32:53.846 INFO:tasks.workunit.client.0.smithi053.stdout: 10 78771 20.60 MB/sec execute 37 sec latency 475.936 ms 2024-09-19T06:32:54.027 INFO:tasks.workunit.client.1.smithi155.stdout: 10 21600 7.69 MB/sec execute 40 sec latency 210.166 ms 2024-09-19T06:32:54.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:54 smithi053 ceph-mon[27968]: pgmap v702: 129 pgs: 129 active+clean; 418 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 4.2 KiB/s rd, 4.1 MiB/s wr, 130 op/s 2024-09-19T06:32:54.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:54 smithi155 ceph-mon[35292]: pgmap v702: 129 pgs: 129 active+clean; 418 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 4.2 KiB/s rd, 4.1 MiB/s wr, 130 op/s 2024-09-19T06:32:54.846 INFO:tasks.workunit.client.0.smithi053.stdout: 10 78802 20.08 MB/sec execute 38 sec latency 478.421 ms 2024-09-19T06:32:55.027 INFO:tasks.workunit.client.1.smithi155.stdout: 10 21685 7.53 MB/sec execute 41 sec latency 152.568 ms 2024-09-19T06:32:55.846 INFO:tasks.workunit.client.0.smithi053.stdout: 10 78823 19.58 MB/sec execute 39 sec latency 526.818 ms 2024-09-19T06:32:56.027 INFO:tasks.workunit.client.1.smithi155.stdout: 10 21792 7.38 MB/sec execute 42 sec latency 110.271 ms 2024-09-19T06:32:56.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:56 smithi053 ceph-mon[27968]: pgmap v703: 129 pgs: 129 active+clean; 419 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 3.9 MiB/s wr, 92 op/s 2024-09-19T06:32:56.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:56 smithi155 ceph-mon[35292]: pgmap v703: 129 pgs: 129 active+clean; 419 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 3.9 MiB/s wr, 92 op/s 2024-09-19T06:32:56.846 INFO:tasks.workunit.client.0.smithi053.stdout: 10 78841 19.10 MB/sec execute 40 sec latency 395.355 ms 2024-09-19T06:32:57.027 INFO:tasks.workunit.client.1.smithi155.stdout: 10 21864 7.21 MB/sec execute 43 sec latency 181.752 ms 2024-09-19T06:32:57.846 INFO:tasks.workunit.client.0.smithi053.stdout: 10 79045 18.75 MB/sec execute 41 sec latency 435.250 ms 2024-09-19T06:32:58.027 INFO:tasks.workunit.client.1.smithi155.stdout: 10 21990 7.08 MB/sec execute 44 sec latency 168.283 ms 2024-09-19T06:32:58.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:58 smithi053 ceph-mon[27968]: pgmap v704: 129 pgs: 129 active+clean; 419 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 3.7 MiB/s wr, 88 op/s 2024-09-19T06:32:58.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:58 smithi155 ceph-mon[35292]: pgmap v704: 129 pgs: 129 active+clean; 419 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 3.7 MiB/s wr, 88 op/s 2024-09-19T06:32:58.846 INFO:tasks.workunit.client.0.smithi053.stdout: 10 80591 20.03 MB/sec execute 42 sec latency 113.501 ms 2024-09-19T06:32:59.027 INFO:tasks.workunit.client.1.smithi155.stdout: 10 22139 7.00 MB/sec execute 45 sec latency 124.358 ms 2024-09-19T06:32:59.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:32:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:32:59.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:32:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:32:59.846 INFO:tasks.workunit.client.0.smithi053.stdout: 10 81867 21.60 MB/sec execute 43 sec latency 126.265 ms 2024-09-19T06:33:00.027 INFO:tasks.workunit.client.1.smithi155.stdout: 10 22289 6.98 MB/sec execute 46 sec latency 95.697 ms 2024-09-19T06:33:00.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:00 smithi053 ceph-mon[27968]: pgmap v705: 129 pgs: 129 active+clean; 423 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 6.4 KiB/s rd, 3.8 MiB/s wr, 98 op/s 2024-09-19T06:33:00.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:00 smithi155 ceph-mon[35292]: pgmap v705: 129 pgs: 129 active+clean; 423 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 6.4 KiB/s rd, 3.8 MiB/s wr, 98 op/s 2024-09-19T06:33:00.846 INFO:tasks.workunit.client.0.smithi053.stdout: 10 82437 21.73 MB/sec execute 44 sec latency 112.676 ms 2024-09-19T06:33:01.027 INFO:tasks.workunit.client.1.smithi155.stdout: 10 22437 7.02 MB/sec execute 47 sec latency 116.066 ms 2024-09-19T06:33:01.846 INFO:tasks.workunit.client.0.smithi053.stdout: 10 83198 22.28 MB/sec execute 45 sec latency 101.178 ms 2024-09-19T06:33:02.028 INFO:tasks.workunit.client.1.smithi155.stdout: 10 22609 7.14 MB/sec execute 48 sec latency 98.894 ms 2024-09-19T06:33:02.648 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:02 smithi053 ceph-mon[27968]: pgmap v706: 129 pgs: 129 active+clean; 422 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.7 MiB/s wr, 91 op/s 2024-09-19T06:33:02.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:02 smithi155 ceph-mon[35292]: pgmap v706: 129 pgs: 129 active+clean; 422 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 4.9 KiB/s rd, 2.7 MiB/s wr, 91 op/s 2024-09-19T06:33:02.846 INFO:tasks.workunit.client.0.smithi053.stdout: 10 84508 23.27 MB/sec execute 46 sec latency 121.093 ms 2024-09-19T06:33:03.027 INFO:tasks.workunit.client.1.smithi155.stdout: 10 22798 7.37 MB/sec execute 49 sec latency 92.982 ms 2024-09-19T06:33:03.846 INFO:tasks.workunit.client.0.smithi053.stdout: 10 85859 24.35 MB/sec execute 47 sec latency 111.076 ms 2024-09-19T06:33:04.028 INFO:tasks.workunit.client.1.smithi155.stdout: 10 22963 7.25 MB/sec execute 50 sec latency 85.822 ms 2024-09-19T06:33:04.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:04 smithi053 ceph-mon[27968]: pgmap v707: 129 pgs: 129 active+clean; 405 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 9.4 KiB/s rd, 3.7 MiB/s wr, 164 op/s 2024-09-19T06:33:04.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:04 smithi155 ceph-mon[35292]: pgmap v707: 129 pgs: 129 active+clean; 405 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 9.4 KiB/s rd, 3.7 MiB/s wr, 164 op/s 2024-09-19T06:33:04.846 INFO:tasks.workunit.client.0.smithi053.stdout: 10 85983 23.91 MB/sec execute 48 sec latency 202.608 ms 2024-09-19T06:33:05.028 INFO:tasks.workunit.client.1.smithi155.stdout: 10 23143 7.13 MB/sec execute 51 sec latency 150.114 ms 2024-09-19T06:33:05.846 INFO:tasks.workunit.client.0.smithi053.stdout: 10 86941 24.40 MB/sec execute 49 sec latency 103.993 ms 2024-09-19T06:33:06.028 INFO:tasks.workunit.client.1.smithi155.stdout: 10 23311 7.03 MB/sec execute 52 sec latency 108.740 ms 2024-09-19T06:33:06.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:06 smithi155 ceph-mon[35292]: pgmap v708: 129 pgs: 129 active+clean; 404 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 8.2 KiB/s rd, 3.3 MiB/s wr, 169 op/s 2024-09-19T06:33:06.846 INFO:tasks.workunit.client.0.smithi053.stdout: 10 88293 25.48 MB/sec execute 50 sec latency 110.958 ms 2024-09-19T06:33:06.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:06 smithi053 ceph-mon[27968]: pgmap v708: 129 pgs: 129 active+clean; 404 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 8.2 KiB/s rd, 3.3 MiB/s wr, 169 op/s 2024-09-19T06:33:07.028 INFO:tasks.workunit.client.1.smithi155.stdout: 10 23500 6.99 MB/sec execute 53 sec latency 88.147 ms 2024-09-19T06:33:07.846 INFO:tasks.workunit.client.0.smithi053.stdout: 10 89440 26.21 MB/sec execute 51 sec latency 111.517 ms 2024-09-19T06:33:08.028 INFO:tasks.workunit.client.1.smithi155.stdout: 10 23677 7.02 MB/sec execute 54 sec latency 86.773 ms 2024-09-19T06:33:08.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:08 smithi155 ceph-mon[35292]: pgmap v709: 129 pgs: 129 active+clean; 404 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 8.2 KiB/s rd, 2.8 MiB/s wr, 175 op/s 2024-09-19T06:33:08.846 INFO:tasks.workunit.client.0.smithi053.stdout: 10 89634 25.80 MB/sec execute 52 sec latency 91.905 ms 2024-09-19T06:33:08.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:08 smithi053 ceph-mon[27968]: pgmap v709: 129 pgs: 129 active+clean; 404 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 8.2 KiB/s rd, 2.8 MiB/s wr, 175 op/s 2024-09-19T06:33:09.028 INFO:tasks.workunit.client.1.smithi155.stdout: 10 23800 6.99 MB/sec execute 55 sec latency 75.616 ms 2024-09-19T06:33:09.847 INFO:tasks.workunit.client.0.smithi053.stdout: 10 91096 26.48 MB/sec execute 53 sec latency 187.324 ms 2024-09-19T06:33:10.028 INFO:tasks.workunit.client.1.smithi155.stdout: 10 23977 7.18 MB/sec execute 56 sec latency 153.884 ms 2024-09-19T06:33:10.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:10 smithi155 ceph-mon[35292]: pgmap v710: 129 pgs: 129 active+clean; 404 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 2.8 MiB/s wr, 202 op/s 2024-09-19T06:33:10.846 INFO:tasks.workunit.client.0.smithi053.stdout: 10 91971 27.27 MB/sec execute 54 sec latency 403.723 ms 2024-09-19T06:33:10.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:10 smithi053 ceph-mon[27968]: pgmap v710: 129 pgs: 129 active+clean; 404 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 2.8 MiB/s wr, 202 op/s 2024-09-19T06:33:11.028 INFO:tasks.workunit.client.1.smithi155.stdout: 10 24213 7.51 MB/sec execute 57 sec latency 110.170 ms 2024-09-19T06:33:11.847 INFO:tasks.workunit.client.0.smithi053.stdout: 10 93006 27.79 MB/sec execute 55 sec latency 115.611 ms 2024-09-19T06:33:12.028 INFO:tasks.workunit.client.1.smithi155.stdout: 10 24464 7.65 MB/sec execute 58 sec latency 101.253 ms 2024-09-19T06:33:12.647 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:12 smithi053 ceph-mon[27968]: pgmap v711: 129 pgs: 129 active+clean; 407 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 6.3 KiB/s rd, 2.5 MiB/s wr, 186 op/s 2024-09-19T06:33:12.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:12 smithi155 ceph-mon[35292]: pgmap v711: 129 pgs: 129 active+clean; 407 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 6.3 KiB/s rd, 2.5 MiB/s wr, 186 op/s 2024-09-19T06:33:12.847 INFO:tasks.workunit.client.0.smithi053.stdout: 10 93188 27.37 MB/sec execute 56 sec latency 123.912 ms 2024-09-19T06:33:13.028 INFO:tasks.workunit.client.1.smithi155.stdout: 10 24616 7.70 MB/sec execute 59 sec latency 174.722 ms 2024-09-19T06:33:13.847 INFO:tasks.workunit.client.0.smithi053.stdout: 10 94206 27.73 MB/sec execute 57 sec latency 312.444 ms 2024-09-19T06:33:14.028 INFO:tasks.workunit.client.1.smithi155.stdout: 10 24641 7.57 MB/sec execute 60 sec latency 178.408 ms 2024-09-19T06:33:14.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:14 smithi155 ceph-mon[35292]: pgmap v712: 129 pgs: 129 active+clean; 425 MiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 8.2 KiB/s rd, 4.5 MiB/s wr, 231 op/s 2024-09-19T06:33:14.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:14 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:33:14.847 INFO:tasks.workunit.client.0.smithi053.stdout: 10 95425 28.59 MB/sec execute 58 sec latency 132.081 ms 2024-09-19T06:33:14.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:14 smithi053 ceph-mon[27968]: pgmap v712: 129 pgs: 129 active+clean; 425 MiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 8.2 KiB/s rd, 4.5 MiB/s wr, 231 op/s 2024-09-19T06:33:14.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:14 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:33:15.028 INFO:tasks.workunit.client.1.smithi155.stdout: 10 24724 7.60 MB/sec execute 61 sec latency 99.254 ms 2024-09-19T06:33:15.847 INFO:tasks.workunit.client.0.smithi053.stdout: 10 96512 29.13 MB/sec execute 59 sec latency 168.626 ms 2024-09-19T06:33:16.028 INFO:tasks.workunit.client.1.smithi155.stdout: 10 24915 7.85 MB/sec execute 62 sec latency 94.218 ms 2024-09-19T06:33:16.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:16 smithi155 ceph-mon[35292]: pgmap v713: 129 pgs: 129 active+clean; 430 MiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.9 MiB/s wr, 164 op/s 2024-09-19T06:33:16.847 INFO:tasks.workunit.client.0.smithi053.stdout: 10 96665 28.70 MB/sec execute 60 sec latency 117.377 ms 2024-09-19T06:33:16.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:16 smithi053 ceph-mon[27968]: pgmap v713: 129 pgs: 129 active+clean; 430 MiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.9 MiB/s wr, 164 op/s 2024-09-19T06:33:17.028 INFO:tasks.workunit.client.1.smithi155.stdout: 10 25074 7.83 MB/sec execute 63 sec latency 97.939 ms 2024-09-19T06:33:17.847 INFO:tasks.workunit.client.0.smithi053.stdout: 10 97799 29.07 MB/sec execute 61 sec latency 114.337 ms 2024-09-19T06:33:18.028 INFO:tasks.workunit.client.1.smithi155.stdout: 10 25197 7.80 MB/sec execute 64 sec latency 89.614 ms 2024-09-19T06:33:18.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:18 smithi155 ceph-mon[35292]: pgmap v714: 129 pgs: 129 active+clean; 430 MiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.8 MiB/s wr, 124 op/s 2024-09-19T06:33:18.847 INFO:tasks.workunit.client.0.smithi053.stdout: 10 98675 29.58 MB/sec execute 62 sec latency 325.294 ms 2024-09-19T06:33:18.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:18 smithi053 ceph-mon[27968]: pgmap v714: 129 pgs: 129 active+clean; 430 MiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.8 MiB/s wr, 124 op/s 2024-09-19T06:33:19.028 INFO:tasks.workunit.client.1.smithi155.stdout: 10 25287 7.71 MB/sec execute 65 sec latency 175.137 ms 2024-09-19T06:33:19.847 INFO:tasks.workunit.client.0.smithi053.stdout: 10 98858 29.28 MB/sec execute 63 sec latency 395.486 ms 2024-09-19T06:33:20.028 INFO:tasks.workunit.client.1.smithi155.stdout: 10 25359 7.59 MB/sec execute 66 sec latency 155.884 ms 2024-09-19T06:33:20.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:20 smithi155 ceph-mon[35292]: pgmap v715: 129 pgs: 129 active+clean; 435 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 5.2 KiB/s rd, 4.7 MiB/s wr, 151 op/s 2024-09-19T06:33:20.847 INFO:tasks.workunit.client.0.smithi053.stdout: 10 99118 29.06 MB/sec execute 64 sec latency 527.776 ms 2024-09-19T06:33:20.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:20 smithi053 ceph-mon[27968]: pgmap v715: 129 pgs: 129 active+clean; 435 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 5.2 KiB/s rd, 4.7 MiB/s wr, 151 op/s 2024-09-19T06:33:21.028 INFO:tasks.workunit.client.1.smithi155.stdout: 10 25435 7.49 MB/sec execute 67 sec latency 199.193 ms 2024-09-19T06:33:21.847 INFO:tasks.workunit.client.0.smithi053.stdout: 10 99326 28.77 MB/sec execute 65 sec latency 475.631 ms 2024-09-19T06:33:22.028 INFO:tasks.workunit.client.1.smithi155.stdout: 10 25517 7.39 MB/sec execute 68 sec latency 262.901 ms 2024-09-19T06:33:22.647 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:22 smithi053 ceph-mon[27968]: pgmap v716: 129 pgs: 129 active+clean; 436 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 3.4 KiB/s rd, 4.8 MiB/s wr, 123 op/s 2024-09-19T06:33:22.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:22 smithi155 ceph-mon[35292]: pgmap v716: 129 pgs: 129 active+clean; 436 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 3.4 KiB/s rd, 4.8 MiB/s wr, 123 op/s 2024-09-19T06:33:22.847 INFO:tasks.workunit.client.0.smithi053.stdout: 10 100039 28.93 MB/sec execute 66 sec latency 511.252 ms 2024-09-19T06:33:23.028 INFO:tasks.workunit.client.1.smithi155.stdout: 10 25612 7.31 MB/sec execute 69 sec latency 200.810 ms 2024-09-19T06:33:23.847 INFO:tasks.workunit.client.0.smithi053.stdout: 10 100046 28.50 MB/sec execute 67 sec latency 484.047 ms 2024-09-19T06:33:24.028 INFO:tasks.workunit.client.1.smithi155.stdout: 10 25719 7.24 MB/sec execute 70 sec latency 182.544 ms 2024-09-19T06:33:24.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:24 smithi155 ceph-mon[35292]: pgmap v717: 129 pgs: 129 active+clean; 436 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 8.2 KiB/s rd, 5.0 MiB/s wr, 142 op/s 2024-09-19T06:33:24.847 INFO:tasks.workunit.client.0.smithi053.stdout: 10 100063 28.09 MB/sec execute 68 sec latency 458.839 ms 2024-09-19T06:33:24.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:24 smithi053 ceph-mon[27968]: pgmap v717: 129 pgs: 129 active+clean; 436 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 8.2 KiB/s rd, 5.0 MiB/s wr, 142 op/s 2024-09-19T06:33:25.028 INFO:tasks.workunit.client.1.smithi155.stdout: 10 25816 7.21 MB/sec execute 71 sec latency 197.889 ms 2024-09-19T06:33:25.847 INFO:tasks.workunit.client.0.smithi053.stdout: 10 100097 27.70 MB/sec execute 69 sec latency 408.873 ms 2024-09-19T06:33:26.028 INFO:tasks.workunit.client.1.smithi155.stdout: 10 25903 7.16 MB/sec execute 72 sec latency 263.400 ms 2024-09-19T06:33:26.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:26 smithi155 ceph-mon[35292]: pgmap v718: 129 pgs: 129 active+clean; 433 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 6.2 KiB/s rd, 3.0 MiB/s wr, 101 op/s 2024-09-19T06:33:26.847 INFO:tasks.workunit.client.0.smithi053.stdout: 10 100107 27.30 MB/sec execute 70 sec latency 575.616 ms 2024-09-19T06:33:26.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:26 smithi053 ceph-mon[27968]: pgmap v718: 129 pgs: 129 active+clean; 433 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 6.2 KiB/s rd, 3.0 MiB/s wr, 101 op/s 2024-09-19T06:33:27.028 INFO:tasks.workunit.client.1.smithi155.stdout: 10 26016 7.14 MB/sec execute 73 sec latency 203.056 ms 2024-09-19T06:33:27.847 INFO:tasks.workunit.client.0.smithi053.stdout: 10 100163 26.94 MB/sec execute 71 sec latency 418.724 ms 2024-09-19T06:33:28.028 INFO:tasks.workunit.client.1.smithi155.stdout: 10 26189 7.22 MB/sec execute 74 sec latency 122.292 ms 2024-09-19T06:33:28.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:27 smithi155 ceph-mon[35292]: pgmap v719: 129 pgs: 129 active+clean; 429 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 6.2 KiB/s rd, 2.8 MiB/s wr, 109 op/s 2024-09-19T06:33:28.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:27 smithi053 ceph-mon[27968]: pgmap v719: 129 pgs: 129 active+clean; 429 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 6.2 KiB/s rd, 2.8 MiB/s wr, 109 op/s 2024-09-19T06:33:28.847 INFO:tasks.workunit.client.0.smithi053.stdout: 10 100170 26.57 MB/sec execute 72 sec latency 437.898 ms 2024-09-19T06:33:29.028 INFO:tasks.workunit.client.1.smithi155.stdout: 10 26374 7.37 MB/sec execute 75 sec latency 159.366 ms 2024-09-19T06:33:29.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:28 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:33:29.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:28 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:33:29.847 INFO:tasks.workunit.client.0.smithi053.stdout: 10 100199 26.21 MB/sec execute 73 sec latency 409.004 ms 2024-09-19T06:33:30.028 INFO:tasks.workunit.client.1.smithi155.stdout: 10 26534 7.29 MB/sec execute 76 sec latency 150.484 ms 2024-09-19T06:33:30.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:29 smithi155 ceph-mon[35292]: pgmap v720: 129 pgs: 129 active+clean; 419 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 9.2 KiB/s rd, 3.1 MiB/s wr, 164 op/s 2024-09-19T06:33:30.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:29 smithi053 ceph-mon[27968]: pgmap v720: 129 pgs: 129 active+clean; 419 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 9.2 KiB/s rd, 3.1 MiB/s wr, 164 op/s 2024-09-19T06:33:30.847 INFO:tasks.workunit.client.0.smithi053.stdout: 10 100298 25.89 MB/sec execute 74 sec latency 401.116 ms 2024-09-19T06:33:31.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 26669 7.20 MB/sec execute 77 sec latency 172.656 ms 2024-09-19T06:33:31.847 INFO:tasks.workunit.client.0.smithi053.stdout: 10 100465 25.67 MB/sec execute 75 sec latency 501.432 ms 2024-09-19T06:33:32.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 26834 7.13 MB/sec execute 78 sec latency 167.310 ms 2024-09-19T06:33:32.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:32 smithi053 ceph-mon[27968]: pgmap v721: 129 pgs: 129 active+clean; 418 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 7.7 KiB/s rd, 2.4 MiB/s wr, 132 op/s 2024-09-19T06:33:32.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:32 smithi155 ceph-mon[35292]: pgmap v721: 129 pgs: 129 active+clean; 418 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 7.7 KiB/s rd, 2.4 MiB/s wr, 132 op/s 2024-09-19T06:33:32.847 INFO:tasks.workunit.client.0.smithi053.stdout: 10 100919 25.78 MB/sec execute 76 sec latency 492.992 ms 2024-09-19T06:33:33.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 26976 7.08 MB/sec execute 79 sec latency 172.728 ms 2024-09-19T06:33:33.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:33 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:33:33.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:33 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:33:33.848 INFO:tasks.workunit.client.0.smithi053.stdout: 10 101076 25.46 MB/sec execute 77 sec latency 668.524 ms 2024-09-19T06:33:34.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 27128 7.06 MB/sec execute 80 sec latency 121.714 ms 2024-09-19T06:33:34.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:34 smithi053 ceph-mon[27968]: pgmap v722: 129 pgs: 129 active+clean; 408 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 9.4 KiB/s rd, 2.4 MiB/s wr, 192 op/s 2024-09-19T06:33:34.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:34 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:34.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:34 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:33:34.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:34 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:33:34.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:34 smithi155 ceph-mon[35292]: pgmap v722: 129 pgs: 129 active+clean; 408 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 9.4 KiB/s rd, 2.4 MiB/s wr, 192 op/s 2024-09-19T06:33:34.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:34 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:34.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:34 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:33:34.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:34 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:33:34.848 INFO:tasks.workunit.client.0.smithi053.stdout: 10 101345 25.19 MB/sec execute 78 sec latency 516.288 ms 2024-09-19T06:33:35.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 27245 7.06 MB/sec execute 81 sec latency 164.241 ms 2024-09-19T06:33:35.848 INFO:tasks.workunit.client.0.smithi053.stdout: 10 101742 25.04 MB/sec execute 79 sec latency 555.117 ms 2024-09-19T06:33:36.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 27357 7.03 MB/sec execute 82 sec latency 159.063 ms 2024-09-19T06:33:36.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:36 smithi053 ceph-mon[27968]: pgmap v723: 129 pgs: 129 active+clean; 408 MiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 4.7 KiB/s rd, 1.9 MiB/s wr, 165 op/s 2024-09-19T06:33:36.484 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:36 smithi155 ceph-mon[35292]: pgmap v723: 129 pgs: 129 active+clean; 408 MiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 4.7 KiB/s rd, 1.9 MiB/s wr, 165 op/s 2024-09-19T06:33:36.848 INFO:tasks.workunit.client.0.smithi053.stdout: 10 102070 25.14 MB/sec execute 80 sec latency 478.810 ms 2024-09-19T06:33:37.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 27499 7.14 MB/sec execute 83 sec latency 179.434 ms 2024-09-19T06:33:37.848 INFO:tasks.workunit.client.0.smithi053.stdout: 10 103304 25.75 MB/sec execute 81 sec latency 339.134 ms 2024-09-19T06:33:38.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 27726 7.33 MB/sec execute 84 sec latency 105.280 ms 2024-09-19T06:33:38.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:38 smithi053 ceph-mon[27968]: pgmap v724: 129 pgs: 129 active+clean; 402 MiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 4.7 KiB/s rd, 2.3 MiB/s wr, 166 op/s 2024-09-19T06:33:38.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:38 smithi155 ceph-mon[35292]: pgmap v724: 129 pgs: 129 active+clean; 402 MiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 4.7 KiB/s rd, 2.3 MiB/s wr, 166 op/s 2024-09-19T06:33:38.848 INFO:tasks.workunit.client.0.smithi053.stdout: 10 103693 25.65 MB/sec execute 82 sec latency 136.135 ms 2024-09-19T06:33:39.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 27992 7.47 MB/sec execute 85 sec latency 114.650 ms 2024-09-19T06:33:39.848 INFO:tasks.workunit.client.0.smithi053.stdout: 10 104164 25.65 MB/sec execute 83 sec latency 129.913 ms 2024-09-19T06:33:40.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 28165 7.50 MB/sec execute 86 sec latency 118.988 ms 2024-09-19T06:33:40.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:40 smithi053 ceph-mon[27968]: pgmap v725: 129 pgs: 129 active+clean; 406 MiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 6.2 KiB/s rd, 3.1 MiB/s wr, 191 op/s 2024-09-19T06:33:40.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:40 smithi155 ceph-mon[35292]: pgmap v725: 129 pgs: 129 active+clean; 406 MiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 6.2 KiB/s rd, 3.1 MiB/s wr, 191 op/s 2024-09-19T06:33:40.848 INFO:tasks.workunit.client.0.smithi053.stdout: 10 105529 26.10 MB/sec execute 84 sec latency 110.944 ms 2024-09-19T06:33:41.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 28190 7.42 MB/sec execute 87 sec latency 96.898 ms 2024-09-19T06:33:41.848 INFO:tasks.workunit.client.0.smithi053.stdout: 10 106863 26.79 MB/sec execute 85 sec latency 115.689 ms 2024-09-19T06:33:42.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 28277 7.44 MB/sec execute 88 sec latency 93.203 ms 2024-09-19T06:33:42.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:42 smithi053 ceph-mon[27968]: pgmap v726: 129 pgs: 129 active+clean; 411 MiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 2.9 MiB/s wr, 134 op/s 2024-09-19T06:33:42.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:42 smithi155 ceph-mon[35292]: pgmap v726: 129 pgs: 129 active+clean; 411 MiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 2.9 MiB/s wr, 134 op/s 2024-09-19T06:33:42.848 INFO:tasks.workunit.client.0.smithi053.stdout: 10 107135 26.65 MB/sec execute 86 sec latency 467.199 ms 2024-09-19T06:33:43.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 28377 7.45 MB/sec execute 89 sec latency 183.580 ms 2024-09-19T06:33:43.848 INFO:tasks.workunit.client.0.smithi053.stdout: 10 107171 26.36 MB/sec execute 87 sec latency 417.181 ms 2024-09-19T06:33:44.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 28526 7.57 MB/sec execute 90 sec latency 144.745 ms 2024-09-19T06:33:44.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:44 smithi053 ceph-mon[27968]: pgmap v727: 129 pgs: 129 active+clean; 421 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 5.0 KiB/s rd, 4.0 MiB/s wr, 164 op/s 2024-09-19T06:33:44.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:44 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:33:44.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:44 smithi155 ceph-mon[35292]: pgmap v727: 129 pgs: 129 active+clean; 421 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 5.0 KiB/s rd, 4.0 MiB/s wr, 164 op/s 2024-09-19T06:33:44.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:44 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:33:44.848 INFO:tasks.workunit.client.0.smithi053.stdout: 10 107189 26.06 MB/sec execute 88 sec latency 384.011 ms 2024-09-19T06:33:45.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 28610 7.52 MB/sec execute 91 sec latency 233.770 ms 2024-09-19T06:33:45.848 INFO:tasks.workunit.client.0.smithi053.stdout: 10 107222 25.78 MB/sec execute 89 sec latency 425.434 ms 2024-09-19T06:33:46.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 28705 7.51 MB/sec execute 92 sec latency 177.499 ms 2024-09-19T06:33:46.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:46 smithi053 ceph-mon[27968]: pgmap v728: 129 pgs: 129 active+clean; 422 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 3.3 KiB/s rd, 3.9 MiB/s wr, 107 op/s 2024-09-19T06:33:46.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:46 smithi155 ceph-mon[35292]: pgmap v728: 129 pgs: 129 active+clean; 422 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 3.3 KiB/s rd, 3.9 MiB/s wr, 107 op/s 2024-09-19T06:33:46.848 INFO:tasks.workunit.client.0.smithi053.stdout: 10 107255 25.50 MB/sec execute 90 sec latency 425.706 ms 2024-09-19T06:33:47.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 28764 7.43 MB/sec execute 93 sec latency 176.769 ms 2024-09-19T06:33:47.848 INFO:tasks.workunit.client.0.smithi053.stdout: 10 107268 25.22 MB/sec execute 91 sec latency 467.001 ms 2024-09-19T06:33:48.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 28835 7.37 MB/sec execute 94 sec latency 213.761 ms 2024-09-19T06:33:48.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:48 smithi053 ceph-mon[27968]: pgmap v729: 129 pgs: 129 active+clean; 424 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 3.3 KiB/s rd, 4.4 MiB/s wr, 109 op/s 2024-09-19T06:33:48.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:48 smithi155 ceph-mon[35292]: pgmap v729: 129 pgs: 129 active+clean; 424 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 3.3 KiB/s rd, 4.4 MiB/s wr, 109 op/s 2024-09-19T06:33:48.848 INFO:tasks.workunit.client.0.smithi053.stdout: 10 107317 24.96 MB/sec execute 92 sec latency 468.107 ms 2024-09-19T06:33:49.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 28894 7.29 MB/sec execute 95 sec latency 182.618 ms 2024-09-19T06:33:49.848 INFO:tasks.workunit.client.0.smithi053.stdout: 10 107388 24.71 MB/sec execute 93 sec latency 425.934 ms 2024-09-19T06:33:50.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 28962 7.22 MB/sec execute 96 sec latency 319.929 ms 2024-09-19T06:33:50.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:50 smithi155 ceph-mon[35292]: pgmap v730: 129 pgs: 129 active+clean; 427 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 6.2 KiB/s rd, 4.5 MiB/s wr, 118 op/s 2024-09-19T06:33:50.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:50 smithi053 ceph-mon[27968]: pgmap v730: 129 pgs: 129 active+clean; 427 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 6.2 KiB/s rd, 4.5 MiB/s wr, 118 op/s 2024-09-19T06:33:50.848 INFO:tasks.workunit.client.0.smithi053.stdout: 10 107689 24.68 MB/sec execute 94 sec latency 412.679 ms 2024-09-19T06:33:51.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 29078 7.16 MB/sec execute 97 sec latency 169.086 ms 2024-09-19T06:33:51.848 INFO:tasks.workunit.client.0.smithi053.stdout: 10 108266 24.67 MB/sec execute 95 sec latency 441.305 ms 2024-09-19T06:33:52.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 29174 7.10 MB/sec execute 98 sec latency 220.774 ms 2024-09-19T06:33:52.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:52 smithi155 ceph-mon[35292]: pgmap v731: 129 pgs: 129 active+clean; 428 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 4.7 KiB/s rd, 3.6 MiB/s wr, 79 op/s 2024-09-19T06:33:52.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:52 smithi053 ceph-mon[27968]: pgmap v731: 129 pgs: 129 active+clean; 428 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 4.7 KiB/s rd, 3.6 MiB/s wr, 79 op/s 2024-09-19T06:33:52.848 INFO:tasks.workunit.client.0.smithi053.stdout: 10 109423 25.16 MB/sec execute 96 sec latency 180.216 ms 2024-09-19T06:33:53.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 29332 7.09 MB/sec execute 99 sec latency 96.156 ms 2024-09-19T06:33:53.848 INFO:tasks.workunit.client.0.smithi053.stdout: 10 110695 25.60 MB/sec execute 97 sec latency 203.108 ms 2024-09-19T06:33:54.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 29471 7.07 MB/sec execute 100 sec latency 107.148 ms 2024-09-19T06:33:54.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:54 smithi155 ceph-mon[35292]: pgmap v732: 129 pgs: 129 active+clean; 429 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 7.5 KiB/s rd, 3.7 MiB/s wr, 121 op/s 2024-09-19T06:33:54.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:54 smithi053 ceph-mon[27968]: pgmap v732: 129 pgs: 129 active+clean; 429 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 7.5 KiB/s rd, 3.7 MiB/s wr, 121 op/s 2024-09-19T06:33:54.848 INFO:tasks.workunit.client.0.smithi053.stdout: 10 110834 25.38 MB/sec execute 98 sec latency 92.296 ms 2024-09-19T06:33:55.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 29644 7.15 MB/sec execute 101 sec latency 128.657 ms 2024-09-19T06:33:55.848 INFO:tasks.workunit.client.0.smithi053.stdout: 10 111891 25.61 MB/sec execute 99 sec latency 134.363 ms 2024-09-19T06:33:56.032 INFO:tasks.workunit.client.1.smithi155.stdout: 10 29815 7.17 MB/sec execute 102 sec latency 129.800 ms 2024-09-19T06:33:56.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:56 smithi053 ceph-mon[27968]: pgmap v733: 129 pgs: 129 active+clean; 422 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 5.7 KiB/s rd, 2.5 MiB/s wr, 94 op/s 2024-09-19T06:33:56.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:56 smithi155 ceph-mon[35292]: pgmap v733: 129 pgs: 129 active+clean; 422 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 5.7 KiB/s rd, 2.5 MiB/s wr, 94 op/s 2024-09-19T06:33:56.849 INFO:tasks.workunit.client.0.smithi053.stdout: 10 113224 26.18 MB/sec execute 100 sec latency 106.897 ms 2024-09-19T06:33:57.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 30002 7.23 MB/sec execute 103 sec latency 88.449 ms 2024-09-19T06:33:57.849 INFO:tasks.workunit.client.0.smithi053.stdout: 10 114280 26.49 MB/sec execute 101 sec latency 127.183 ms 2024-09-19T06:33:58.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 30173 7.17 MB/sec execute 104 sec latency 100.485 ms 2024-09-19T06:33:58.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:58 smithi155 ceph-mon[35292]: pgmap v734: 129 pgs: 129 active+clean; 419 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 5.7 KiB/s rd, 2.7 MiB/s wr, 145 op/s 2024-09-19T06:33:58.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:58 smithi053 ceph-mon[27968]: pgmap v734: 129 pgs: 129 active+clean; 419 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 5.7 KiB/s rd, 2.7 MiB/s wr, 145 op/s 2024-09-19T06:33:58.849 INFO:tasks.workunit.client.0.smithi053.stdout: 10 114510 26.30 MB/sec execute 102 sec latency 178.360 ms 2024-09-19T06:33:59.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 30379 7.12 MB/sec execute 105 sec latency 79.503 ms 2024-09-19T06:33:59.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:33:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:33:59.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:33:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:33:59.849 INFO:tasks.workunit.client.0.smithi053.stdout: 10 115907 26.62 MB/sec execute 103 sec latency 327.118 ms 2024-09-19T06:34:00.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 30554 7.08 MB/sec execute 106 sec latency 96.194 ms 2024-09-19T06:34:00.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:00 smithi155 ceph-mon[35292]: pgmap v735: 129 pgs: 129 active+clean; 410 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 8.8 KiB/s rd, 2.7 MiB/s wr, 176 op/s 2024-09-19T06:34:00.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:00 smithi053 ceph-mon[27968]: pgmap v735: 129 pgs: 129 active+clean; 410 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 8.8 KiB/s rd, 2.7 MiB/s wr, 176 op/s 2024-09-19T06:34:00.849 INFO:tasks.workunit.client.0.smithi053.stdout: 10 116854 27.08 MB/sec execute 104 sec latency 121.627 ms 2024-09-19T06:34:01.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 30716 7.08 MB/sec execute 107 sec latency 103.119 ms 2024-09-19T06:34:01.849 INFO:tasks.workunit.client.0.smithi053.stdout: 10 117828 27.30 MB/sec execute 105 sec latency 156.636 ms 2024-09-19T06:34:02.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 30868 7.09 MB/sec execute 108 sec latency 92.041 ms 2024-09-19T06:34:02.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:02 smithi155 ceph-mon[35292]: pgmap v736: 129 pgs: 129 active+clean; 410 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 5.9 KiB/s rd, 2.3 MiB/s wr, 163 op/s 2024-09-19T06:34:02.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:02 smithi053 ceph-mon[27968]: pgmap v736: 129 pgs: 129 active+clean; 410 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 5.9 KiB/s rd, 2.3 MiB/s wr, 163 op/s 2024-09-19T06:34:02.849 INFO:tasks.workunit.client.0.smithi053.stdout: 10 118033 27.09 MB/sec execute 106 sec latency 99.980 ms 2024-09-19T06:34:03.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 31038 7.20 MB/sec execute 109 sec latency 84.445 ms 2024-09-19T06:34:03.849 INFO:tasks.workunit.client.0.smithi053.stdout: 10 119032 27.28 MB/sec execute 107 sec latency 358.678 ms 2024-09-19T06:34:04.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 31244 7.26 MB/sec execute 110 sec latency 129.098 ms 2024-09-19T06:34:04.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:04 smithi155 ceph-mon[35292]: pgmap v737: 129 pgs: 129 active+clean; 417 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 7.5 KiB/s rd, 3.1 MiB/s wr, 221 op/s 2024-09-19T06:34:04.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:04 smithi053 ceph-mon[27968]: pgmap v737: 129 pgs: 129 active+clean; 417 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 7.5 KiB/s rd, 3.1 MiB/s wr, 221 op/s 2024-09-19T06:34:04.849 INFO:tasks.workunit.client.0.smithi053.stdout: 10 120389 27.85 MB/sec execute 108 sec latency 121.147 ms 2024-09-19T06:34:05.029 INFO:tasks.workunit.client.1.smithi155.stdout: 10 31521 7.44 MB/sec execute 111 sec latency 94.994 ms 2024-09-19T06:34:05.849 INFO:tasks.workunit.client.0.smithi053.stdout: 10 121319 28.05 MB/sec execute 109 sec latency 361.597 ms 2024-09-19T06:34:06.030 INFO:tasks.workunit.client.1.smithi155.stdout: 10 31693 7.44 MB/sec execute 112 sec latency 94.835 ms 2024-09-19T06:34:06.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:06 smithi155 ceph-mon[35292]: pgmap v738: 129 pgs: 129 active+clean; 420 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 4.7 KiB/s rd, 3.1 MiB/s wr, 187 op/s 2024-09-19T06:34:06.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:06 smithi053 ceph-mon[27968]: pgmap v738: 129 pgs: 129 active+clean; 420 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 4.7 KiB/s rd, 3.1 MiB/s wr, 187 op/s 2024-09-19T06:34:06.849 INFO:tasks.workunit.client.0.smithi053.stdout: 10 121467 27.82 MB/sec execute 110 sec latency 91.886 ms 2024-09-19T06:34:07.030 INFO:tasks.workunit.client.1.smithi155.stdout: 10 31738 7.40 MB/sec execute 113 sec latency 102.774 ms 2024-09-19T06:34:07.849 INFO:tasks.workunit.client.0.smithi053.stdout: 10 122296 27.99 MB/sec execute 111 sec latency 283.704 ms 2024-09-19T06:34:08.030 INFO:tasks.workunit.client.1.smithi155.stdout: 10 31791 7.37 MB/sec execute 114 sec latency 149.791 ms 2024-09-19T06:34:08.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:08 smithi155 ceph-mon[35292]: pgmap v739: 129 pgs: 129 active+clean; 422 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 4.7 KiB/s rd, 3.2 MiB/s wr, 183 op/s 2024-09-19T06:34:08.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:08 smithi053 ceph-mon[27968]: pgmap v739: 129 pgs: 129 active+clean; 422 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 4.7 KiB/s rd, 3.2 MiB/s wr, 183 op/s 2024-09-19T06:34:08.849 INFO:tasks.workunit.client.0.smithi053.stdout: 10 123465 28.31 MB/sec execute 112 sec latency 226.757 ms 2024-09-19T06:34:09.030 INFO:tasks.workunit.client.1.smithi155.stdout: 10 31936 7.45 MB/sec execute 115 sec latency 130.249 ms 2024-09-19T06:34:09.849 INFO:tasks.workunit.client.0.smithi053.stdout: 10 124866 28.74 MB/sec execute 113 sec latency 116.042 ms 2024-09-19T06:34:10.030 INFO:tasks.workunit.client.1.smithi155.stdout: 10 32104 7.53 MB/sec execute 116 sec latency 129.921 ms 2024-09-19T06:34:10.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:10 smithi155 ceph-mon[35292]: pgmap v740: 129 pgs: 129 active+clean; 434 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 6.7 KiB/s rd, 4.0 MiB/s wr, 153 op/s 2024-09-19T06:34:10.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:10 smithi053 ceph-mon[27968]: pgmap v740: 129 pgs: 129 active+clean; 434 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 6.7 KiB/s rd, 4.0 MiB/s wr, 153 op/s 2024-09-19T06:34:10.849 INFO:tasks.workunit.client.0.smithi053.stdout: 10 124988 28.52 MB/sec execute 114 sec latency 229.048 ms 2024-09-19T06:34:11.030 INFO:tasks.workunit.client.1.smithi155.stdout: 10 32231 7.50 MB/sec execute 117 sec latency 106.008 ms 2024-09-19T06:34:11.849 INFO:tasks.workunit.client.0.smithi053.stdout: 10 125172 28.31 MB/sec execute 115 sec latency 393.331 ms 2024-09-19T06:34:12.030 INFO:tasks.workunit.client.1.smithi155.stdout: 10 32332 7.48 MB/sec execute 118 sec latency 144.462 ms 2024-09-19T06:34:12.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:12 smithi155 ceph-mon[35292]: pgmap v741: 129 pgs: 129 active+clean; 438 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 3.6 KiB/s rd, 3.9 MiB/s wr, 129 op/s 2024-09-19T06:34:12.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:12 smithi053 ceph-mon[27968]: pgmap v741: 129 pgs: 129 active+clean; 438 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 3.6 KiB/s rd, 3.9 MiB/s wr, 129 op/s 2024-09-19T06:34:12.849 INFO:tasks.workunit.client.0.smithi053.stdout: 10 126577 28.59 MB/sec execute 116 sec latency 199.207 ms 2024-09-19T06:34:13.030 INFO:tasks.workunit.client.1.smithi155.stdout: 10 32429 7.43 MB/sec execute 119 sec latency 139.725 ms 2024-09-19T06:34:13.849 INFO:tasks.workunit.client.0.smithi053.stdout: 10 126997 28.72 MB/sec execute 117 sec latency 382.317 ms 2024-09-19T06:34:14.030 INFO:tasks.workunit.client.1.smithi155.stdout: 10 32512 7.37 MB/sec execute 120 sec latency 161.197 ms 2024-09-19T06:34:14.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:14 smithi155 ceph-mon[35292]: pgmap v742: 129 pgs: 129 active+clean; 442 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 6.1 KiB/s rd, 5.0 MiB/s wr, 143 op/s 2024-09-19T06:34:14.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:14 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:34:14.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:14 smithi053 ceph-mon[27968]: pgmap v742: 129 pgs: 129 active+clean; 442 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 6.1 KiB/s rd, 5.0 MiB/s wr, 143 op/s 2024-09-19T06:34:14.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:14 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:34:14.849 INFO:tasks.workunit.client.0.smithi053.stdout: 10 127188 28.58 MB/sec execute 118 sec latency 449.699 ms 2024-09-19T06:34:15.030 INFO:tasks.workunit.client.1.smithi155.stdout: 10 32602 7.32 MB/sec execute 121 sec latency 196.118 ms 2024-09-19T06:34:15.849 INFO:tasks.workunit.client.0.smithi053.stdout: 10 127420 28.43 MB/sec execute 119 sec latency 353.070 ms 2024-09-19T06:34:16.030 INFO:tasks.workunit.client.1.smithi155.stdout: 10 32689 7.27 MB/sec execute 122 sec latency 196.511 ms 2024-09-19T06:34:16.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:16 smithi155 ceph-mon[35292]: pgmap v743: 129 pgs: 129 active+clean; 444 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 4.5 KiB/s rd, 4.4 MiB/s wr, 104 op/s 2024-09-19T06:34:16.525 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:16 smithi053 ceph-mon[27968]: pgmap v743: 129 pgs: 129 active+clean; 444 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 4.5 KiB/s rd, 4.4 MiB/s wr, 104 op/s 2024-09-19T06:34:16.849 INFO:tasks.workunit.client.0.smithi053.stdout: 10 127513 28.25 MB/sec execute 120 sec latency 403.147 ms 2024-09-19T06:34:17.030 INFO:tasks.workunit.client.1.smithi155.stdout: 10 32785 7.23 MB/sec execute 123 sec latency 203.322 ms 2024-09-19T06:34:17.849 INFO:tasks.workunit.client.0.smithi053.stdout: 10 128404 28.42 MB/sec execute 121 sec latency 430.218 ms 2024-09-19T06:34:18.030 INFO:tasks.workunit.client.1.smithi155.stdout: 10 32919 7.22 MB/sec execute 124 sec latency 130.007 ms 2024-09-19T06:34:18.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:18 smithi155 ceph-mon[35292]: pgmap v744: 129 pgs: 129 active+clean; 443 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 4.5 KiB/s rd, 4.0 MiB/s wr, 100 op/s 2024-09-19T06:34:18.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:18 smithi053 ceph-mon[27968]: pgmap v744: 129 pgs: 129 active+clean; 443 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 4.5 KiB/s rd, 4.0 MiB/s wr, 100 op/s 2024-09-19T06:34:18.849 INFO:tasks.workunit.client.0.smithi053.stdout: 10 128410 28.19 MB/sec execute 122 sec latency 436.100 ms 2024-09-19T06:34:19.030 INFO:tasks.workunit.client.1.smithi155.stdout: 10 33027 7.19 MB/sec execute 125 sec latency 180.387 ms 2024-09-19T06:34:19.849 INFO:tasks.workunit.client.0.smithi053.stdout: 10 128428 27.97 MB/sec execute 123 sec latency 477.951 ms 2024-09-19T06:34:20.030 INFO:tasks.workunit.client.1.smithi155.stdout: 10 33137 7.20 MB/sec execute 126 sec latency 179.973 ms 2024-09-19T06:34:20.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:20 smithi155 ceph-mon[35292]: pgmap v745: 129 pgs: 129 active+clean; 435 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 8.7 KiB/s rd, 4.3 MiB/s wr, 146 op/s 2024-09-19T06:34:20.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:20 smithi053 ceph-mon[27968]: pgmap v745: 129 pgs: 129 active+clean; 435 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 8.7 KiB/s rd, 4.3 MiB/s wr, 146 op/s 2024-09-19T06:34:20.849 INFO:tasks.workunit.client.0.smithi053.stdout: 10 128462 27.75 MB/sec execute 124 sec latency 475.928 ms 2024-09-19T06:34:21.030 INFO:tasks.workunit.client.1.smithi155.stdout: 10 33261 7.22 MB/sec execute 127 sec latency 190.436 ms 2024-09-19T06:34:21.850 INFO:tasks.workunit.client.0.smithi053.stdout: 10 128468 27.53 MB/sec execute 125 sec latency 417.099 ms 2024-09-19T06:34:22.030 INFO:tasks.workunit.client.1.smithi155.stdout: 10 33370 7.21 MB/sec execute 128 sec latency 199.989 ms 2024-09-19T06:34:22.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:22 smithi155 ceph-mon[35292]: pgmap v746: 129 pgs: 129 active+clean; 432 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 6.7 KiB/s rd, 3.0 MiB/s wr, 129 op/s 2024-09-19T06:34:22.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:22 smithi053 ceph-mon[27968]: pgmap v746: 129 pgs: 129 active+clean; 432 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 6.7 KiB/s rd, 3.0 MiB/s wr, 129 op/s 2024-09-19T06:34:22.850 INFO:tasks.workunit.client.0.smithi053.stdout: 10 128528 27.32 MB/sec execute 126 sec latency 425.969 ms 2024-09-19T06:34:23.030 INFO:tasks.workunit.client.1.smithi155.stdout: 10 33507 7.25 MB/sec execute 129 sec latency 201.650 ms 2024-09-19T06:34:23.850 INFO:tasks.workunit.client.0.smithi053.stdout: 10 128534 27.10 MB/sec execute 127 sec latency 476.114 ms 2024-09-19T06:34:24.030 INFO:tasks.workunit.client.1.smithi155.stdout: 10 33627 7.20 MB/sec execute 130 sec latency 173.009 ms 2024-09-19T06:34:24.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:24 smithi155 ceph-mon[35292]: pgmap v747: 129 pgs: 129 active+clean; 425 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 9.3 KiB/s rd, 3.3 MiB/s wr, 163 op/s 2024-09-19T06:34:24.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:24 smithi053 ceph-mon[27968]: pgmap v747: 129 pgs: 129 active+clean; 425 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 9.3 KiB/s rd, 3.3 MiB/s wr, 163 op/s 2024-09-19T06:34:24.850 INFO:tasks.workunit.client.0.smithi053.stdout: 10 128549 26.89 MB/sec execute 128 sec latency 458.983 ms 2024-09-19T06:34:25.031 INFO:tasks.workunit.client.1.smithi155.stdout: 10 33738 7.15 MB/sec execute 131 sec latency 214.864 ms 2024-09-19T06:34:25.850 INFO:tasks.workunit.client.0.smithi053.stdout: 10 128625 26.70 MB/sec execute 129 sec latency 429.194 ms 2024-09-19T06:34:26.030 INFO:tasks.workunit.client.1.smithi155.stdout: 10 33876 7.11 MB/sec execute 132 sec latency 184.924 ms 2024-09-19T06:34:26.407 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:26 smithi053 ceph-mon[27968]: pgmap v748: 129 pgs: 129 active+clean; 424 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 6.8 KiB/s rd, 2.3 MiB/s wr, 153 op/s 2024-09-19T06:34:26.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:26 smithi155 ceph-mon[35292]: pgmap v748: 129 pgs: 129 active+clean; 424 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 6.8 KiB/s rd, 2.3 MiB/s wr, 153 op/s 2024-09-19T06:34:26.850 INFO:tasks.workunit.client.0.smithi053.stdout: 10 128717 26.51 MB/sec execute 130 sec latency 400.894 ms 2024-09-19T06:34:27.030 INFO:tasks.workunit.client.1.smithi155.stdout: 10 33976 7.07 MB/sec execute 133 sec latency 163.539 ms 2024-09-19T06:34:27.850 INFO:tasks.workunit.client.0.smithi053.stdout: 10 129260 26.63 MB/sec execute 131 sec latency 515.875 ms 2024-09-19T06:34:28.030 INFO:tasks.workunit.client.1.smithi155.stdout: 10 34136 7.04 MB/sec execute 134 sec latency 171.975 ms 2024-09-19T06:34:28.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:28 smithi155 ceph-mon[35292]: pgmap v749: 129 pgs: 129 active+clean; 422 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 6.8 KiB/s rd, 2.0 MiB/s wr, 161 op/s 2024-09-19T06:34:28.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:28 smithi053 ceph-mon[27968]: pgmap v749: 129 pgs: 129 active+clean; 422 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 6.8 KiB/s rd, 2.0 MiB/s wr, 161 op/s 2024-09-19T06:34:28.850 INFO:tasks.workunit.client.0.smithi053.stdout: 10 129469 26.44 MB/sec execute 132 sec latency 549.437 ms 2024-09-19T06:34:29.031 INFO:tasks.workunit.client.1.smithi155.stdout: 10 34246 7.03 MB/sec execute 135 sec latency 151.403 ms 2024-09-19T06:34:29.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:29 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:34:29.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:34:29.850 INFO:tasks.workunit.client.0.smithi053.stdout: 10 130218 26.42 MB/sec execute 133 sec latency 412.903 ms 2024-09-19T06:34:30.030 INFO:tasks.workunit.client.1.smithi155.stdout: 10 34391 7.04 MB/sec execute 136 sec latency 88.686 ms 2024-09-19T06:34:30.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:30 smithi155 ceph-mon[35292]: pgmap v750: 129 pgs: 129 active+clean; 402 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 8.1 KiB/s rd, 2.1 MiB/s wr, 182 op/s 2024-09-19T06:34:30.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:30 smithi053 ceph-mon[27968]: pgmap v750: 129 pgs: 129 active+clean; 402 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 8.1 KiB/s rd, 2.1 MiB/s wr, 182 op/s 2024-09-19T06:34:30.850 INFO:tasks.workunit.client.0.smithi053.stdout: 10 131687 26.98 MB/sec execute 134 sec latency 112.912 ms 2024-09-19T06:34:31.031 INFO:tasks.workunit.client.1.smithi155.stdout: 10 34528 7.04 MB/sec execute 137 sec latency 114.338 ms 2024-09-19T06:34:31.850 INFO:tasks.workunit.client.0.smithi053.stdout: 10 132036 26.91 MB/sec execute 135 sec latency 225.956 ms 2024-09-19T06:34:32.031 INFO:tasks.workunit.client.1.smithi155.stdout: 10 34770 7.17 MB/sec execute 138 sec latency 90.585 ms 2024-09-19T06:34:32.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:32 smithi053 ceph-mon[27968]: pgmap v751: 129 pgs: 129 active+clean; 402 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 3.9 KiB/s rd, 1.6 MiB/s wr, 133 op/s 2024-09-19T06:34:32.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:32 smithi155 ceph-mon[35292]: pgmap v751: 129 pgs: 129 active+clean; 402 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 3.9 KiB/s rd, 1.6 MiB/s wr, 133 op/s 2024-09-19T06:34:32.850 INFO:tasks.workunit.client.0.smithi053.stdout: 10 132660 26.96 MB/sec execute 136 sec latency 148.898 ms 2024-09-19T06:34:33.031 INFO:tasks.workunit.client.1.smithi155.stdout: 10 35037 7.28 MB/sec execute 139 sec latency 89.595 ms 2024-09-19T06:34:33.850 INFO:tasks.workunit.client.0.smithi053.stdout: 10 134002 27.25 MB/sec execute 137 sec latency 118.189 ms 2024-09-19T06:34:34.031 INFO:tasks.workunit.client.1.smithi155.stdout: 10 35211 7.31 MB/sec execute 140 sec latency 114.590 ms 2024-09-19T06:34:34.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:34 smithi155 ceph-mon[35292]: pgmap v752: 129 pgs: 129 active+clean; 416 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 5.6 KiB/s rd, 3.2 MiB/s wr, 179 op/s 2024-09-19T06:34:34.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:34 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:34:34.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:34 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:34:34.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:34 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:34:34.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:34 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:34:34.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:34 smithi053 ceph-mon[27968]: pgmap v752: 129 pgs: 129 active+clean; 416 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 5.6 KiB/s rd, 3.2 MiB/s wr, 179 op/s 2024-09-19T06:34:34.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:34 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:34:34.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:34 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:34:34.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:34 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:34:34.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:34 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:34:34.850 INFO:tasks.workunit.client.0.smithi053.stdout: 10 134573 27.34 MB/sec execute 138 sec latency 397.745 ms 2024-09-19T06:34:35.031 INFO:tasks.workunit.client.1.smithi155.stdout: 10 35273 7.29 MB/sec execute 141 sec latency 137.374 ms 2024-09-19T06:34:35.850 INFO:tasks.workunit.client.0.smithi053.stdout: 10 134606 27.14 MB/sec execute 139 sec latency 427.931 ms 2024-09-19T06:34:36.031 INFO:tasks.workunit.client.1.smithi155.stdout: 10 35314 7.26 MB/sec execute 142 sec latency 139.838 ms 2024-09-19T06:34:36.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:36 smithi155 ceph-mon[35292]: pgmap v753: 129 pgs: 129 active+clean; 422 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 2.9 KiB/s rd, 3.2 MiB/s wr, 132 op/s 2024-09-19T06:34:36.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:36 smithi053 ceph-mon[27968]: pgmap v753: 129 pgs: 129 active+clean; 422 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 2.9 KiB/s rd, 3.2 MiB/s wr, 132 op/s 2024-09-19T06:34:36.850 INFO:tasks.workunit.client.0.smithi053.stdout: 10 135500 27.30 MB/sec execute 140 sec latency 505.006 ms 2024-09-19T06:34:37.031 INFO:tasks.workunit.client.1.smithi155.stdout: 10 35429 7.30 MB/sec execute 143 sec latency 114.937 ms 2024-09-19T06:34:37.850 INFO:tasks.workunit.client.0.smithi053.stdout: 10 135505 27.11 MB/sec execute 141 sec latency 400.510 ms 2024-09-19T06:34:38.031 INFO:tasks.workunit.client.1.smithi155.stdout: 10 35515 7.28 MB/sec execute 144 sec latency 169.543 ms 2024-09-19T06:34:38.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:38 smithi155 ceph-mon[35292]: pgmap v754: 129 pgs: 129 active+clean; 427 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 2.9 KiB/s rd, 3.6 MiB/s wr, 133 op/s 2024-09-19T06:34:38.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:38 smithi053 ceph-mon[27968]: pgmap v754: 129 pgs: 129 active+clean; 427 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 2.9 KiB/s rd, 3.6 MiB/s wr, 133 op/s 2024-09-19T06:34:38.850 INFO:tasks.workunit.client.0.smithi053.stdout: 10 135544 26.92 MB/sec execute 142 sec latency 445.459 ms 2024-09-19T06:34:39.031 INFO:tasks.workunit.client.1.smithi155.stdout: 10 35631 7.35 MB/sec execute 145 sec latency 314.677 ms 2024-09-19T06:34:39.851 INFO:tasks.workunit.client.0.smithi053.stdout: 10 135559 26.74 MB/sec execute 143 sec latency 425.376 ms 2024-09-19T06:34:40.031 INFO:tasks.workunit.client.1.smithi155.stdout: 10 35722 7.31 MB/sec execute 146 sec latency 201.071 ms 2024-09-19T06:34:40.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:40 smithi155 ceph-mon[35292]: pgmap v755: 129 pgs: 129 active+clean; 428 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 4.6 KiB/s rd, 4.0 MiB/s wr, 125 op/s 2024-09-19T06:34:40.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:40 smithi053 ceph-mon[27968]: pgmap v755: 129 pgs: 129 active+clean; 428 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 4.6 KiB/s rd, 4.0 MiB/s wr, 125 op/s 2024-09-19T06:34:40.850 INFO:tasks.workunit.client.0.smithi053.stdout: 10 135580 26.55 MB/sec execute 144 sec latency 425.918 ms 2024-09-19T06:34:41.031 INFO:tasks.workunit.client.1.smithi155.stdout: 10 35823 7.30 MB/sec execute 147 sec latency 170.307 ms 2024-09-19T06:34:41.851 INFO:tasks.workunit.client.0.smithi053.stdout: 10 135624 26.38 MB/sec execute 145 sec latency 400.843 ms 2024-09-19T06:34:42.031 INFO:tasks.workunit.client.1.smithi155.stdout: 10 35884 7.25 MB/sec execute 148 sec latency 203.564 ms 2024-09-19T06:34:42.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:42 smithi155 ceph-mon[35292]: pgmap v756: 129 pgs: 129 active+clean; 429 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 3.3 KiB/s rd, 4.0 MiB/s wr, 97 op/s 2024-09-19T06:34:42.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:42 smithi053 ceph-mon[27968]: pgmap v756: 129 pgs: 129 active+clean; 429 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 3.3 KiB/s rd, 4.0 MiB/s wr, 97 op/s 2024-09-19T06:34:42.851 INFO:tasks.workunit.client.0.smithi053.stdout: 10 135629 26.20 MB/sec execute 146 sec latency 498.066 ms 2024-09-19T06:34:43.031 INFO:tasks.workunit.client.1.smithi155.stdout: 10 35962 7.21 MB/sec execute 149 sec latency 154.659 ms 2024-09-19T06:34:43.851 INFO:tasks.workunit.client.0.smithi053.stdout: 10 135662 26.02 MB/sec execute 147 sec latency 450.781 ms 2024-09-19T06:34:44.031 INFO:tasks.workunit.client.1.smithi155.stdout: 10 36047 7.17 MB/sec execute 150 sec latency 201.740 ms 2024-09-19T06:34:44.177 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:44 smithi155 ceph-mon[35292]: pgmap v757: 129 pgs: 129 active+clean; 434 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 5.8 KiB/s rd, 4.8 MiB/s wr, 128 op/s 2024-09-19T06:34:44.178 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:44 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:34:44.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:44 smithi053 ceph-mon[27968]: pgmap v757: 129 pgs: 129 active+clean; 434 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 5.8 KiB/s rd, 4.8 MiB/s wr, 128 op/s 2024-09-19T06:34:44.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:44 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:34:44.851 INFO:tasks.workunit.client.0.smithi053.stdout: 10 136712 26.18 MB/sec execute 148 sec latency 410.544 ms 2024-09-19T06:34:45.031 INFO:tasks.workunit.client.1.smithi155.stdout: 10 36161 7.13 MB/sec execute 151 sec latency 114.970 ms 2024-09-19T06:34:45.851 INFO:tasks.workunit.client.0.smithi053.stdout: 10 137895 26.50 MB/sec execute 149 sec latency 173.394 ms 2024-09-19T06:34:46.031 INFO:tasks.workunit.client.1.smithi155.stdout: 10 36302 7.11 MB/sec execute 152 sec latency 110.830 ms 2024-09-19T06:34:46.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:46 smithi155 ceph-mon[35292]: pgmap v758: 129 pgs: 129 active+clean; 434 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 4.2 KiB/s rd, 3.4 MiB/s wr, 77 op/s 2024-09-19T06:34:46.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:46 smithi053 ceph-mon[27968]: pgmap v758: 129 pgs: 129 active+clean; 434 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 4.2 KiB/s rd, 3.4 MiB/s wr, 77 op/s 2024-09-19T06:34:46.851 INFO:tasks.workunit.client.0.smithi053.stdout: 10 139054 26.75 MB/sec execute 150 sec latency 200.809 ms 2024-09-19T06:34:47.031 INFO:tasks.workunit.client.1.smithi155.stdout: 10 36464 7.10 MB/sec execute 153 sec latency 114.781 ms 2024-09-19T06:34:47.851 INFO:tasks.workunit.client.0.smithi053.stdout: 10 139251 26.61 MB/sec execute 151 sec latency 100.551 ms 2024-09-19T06:34:48.031 INFO:tasks.workunit.client.1.smithi155.stdout: 10 36607 7.10 MB/sec execute 154 sec latency 110.773 ms 2024-09-19T06:34:48.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:48 smithi155 ceph-mon[35292]: pgmap v759: 129 pgs: 129 active+clean; 432 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 4.2 KiB/s rd, 2.7 MiB/s wr, 82 op/s 2024-09-19T06:34:48.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:48 smithi053 ceph-mon[27968]: pgmap v759: 129 pgs: 129 active+clean; 432 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 4.2 KiB/s rd, 2.7 MiB/s wr, 82 op/s 2024-09-19T06:34:48.851 INFO:tasks.workunit.client.0.smithi053.stdout: 10 140259 26.74 MB/sec execute 152 sec latency 301.590 ms 2024-09-19T06:34:49.031 INFO:tasks.workunit.client.1.smithi155.stdout: 10 36769 7.14 MB/sec execute 155 sec latency 103.350 ms 2024-09-19T06:34:49.851 INFO:tasks.workunit.client.0.smithi053.stdout: 10 140762 26.68 MB/sec execute 153 sec latency 466.904 ms 2024-09-19T06:34:50.031 INFO:tasks.workunit.client.1.smithi155.stdout: 10 36878 7.14 MB/sec execute 156 sec latency 191.475 ms 2024-09-19T06:34:50.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:50 smithi155 ceph-mon[35292]: pgmap v760: 129 pgs: 129 active+clean; 422 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 9.2 KiB/s rd, 2.9 MiB/s wr, 146 op/s 2024-09-19T06:34:50.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:50 smithi053 ceph-mon[27968]: pgmap v760: 129 pgs: 129 active+clean; 422 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 9.2 KiB/s rd, 2.9 MiB/s wr, 146 op/s 2024-09-19T06:34:50.851 INFO:tasks.workunit.client.0.smithi053.stdout: 10 141005 26.66 MB/sec execute 154 sec latency 356.682 ms 2024-09-19T06:34:51.031 INFO:tasks.workunit.client.1.smithi155.stdout: 10 36982 7.10 MB/sec execute 157 sec latency 178.773 ms 2024-09-19T06:34:51.852 INFO:tasks.workunit.client.0.smithi053.stdout: 10 141320 26.69 MB/sec execute 155 sec latency 581.735 ms 2024-09-19T06:34:52.031 INFO:tasks.workunit.client.1.smithi155.stdout: 10 37123 7.13 MB/sec execute 158 sec latency 156.688 ms 2024-09-19T06:34:52.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:52 smithi155 ceph-mon[35292]: pgmap v761: 129 pgs: 129 active+clean; 418 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 7.6 KiB/s rd, 2.6 MiB/s wr, 137 op/s 2024-09-19T06:34:52.573 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:52 smithi053 ceph-mon[27968]: pgmap v761: 129 pgs: 129 active+clean; 418 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 7.6 KiB/s rd, 2.6 MiB/s wr, 137 op/s 2024-09-19T06:34:52.851 INFO:tasks.workunit.client.0.smithi053.stdout: 10 141494 26.55 MB/sec execute 156 sec latency 385.238 ms 2024-09-19T06:34:53.032 INFO:tasks.workunit.client.1.smithi155.stdout: 10 37242 7.09 MB/sec execute 159 sec latency 184.779 ms 2024-09-19T06:34:53.851 INFO:tasks.workunit.client.0.smithi053.stdout: 10 141683 26.47 MB/sec execute 157 sec latency 421.187 ms 2024-09-19T06:34:54.032 INFO:tasks.workunit.client.1.smithi155.stdout: 10 37351 7.06 MB/sec execute 160 sec latency 177.827 ms 2024-09-19T06:34:54.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:54 smithi155 ceph-mon[35292]: pgmap v762: 129 pgs: 129 active+clean; 416 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 9.4 KiB/s rd, 3.1 MiB/s wr, 190 op/s 2024-09-19T06:34:54.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:54 smithi053 ceph-mon[27968]: pgmap v762: 129 pgs: 129 active+clean; 416 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 9.4 KiB/s rd, 3.1 MiB/s wr, 190 op/s 2024-09-19T06:34:54.851 INFO:tasks.workunit.client.0.smithi053.stdout: 10 142651 26.62 MB/sec execute 158 sec latency 369.543 ms 2024-09-19T06:34:55.031 INFO:tasks.workunit.client.1.smithi155.stdout: 10 37547 7.03 MB/sec execute 161 sec latency 70.427 ms 2024-09-19T06:34:55.851 INFO:tasks.workunit.client.0.smithi053.stdout: 10 142869 26.48 MB/sec execute 159 sec latency 174.836 ms 2024-09-19T06:34:56.032 INFO:tasks.workunit.client.1.smithi155.stdout: 10 37739 7.03 MB/sec execute 162 sec latency 83.115 ms 2024-09-19T06:34:56.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:56 smithi155 ceph-mon[35292]: pgmap v763: 129 pgs: 129 active+clean; 416 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 6.9 KiB/s rd, 2.3 MiB/s wr, 163 op/s 2024-09-19T06:34:56.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:56 smithi053 ceph-mon[27968]: pgmap v763: 129 pgs: 129 active+clean; 416 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 6.9 KiB/s rd, 2.3 MiB/s wr, 163 op/s 2024-09-19T06:34:56.851 INFO:tasks.workunit.client.0.smithi053.stdout: 10 144094 26.66 MB/sec execute 160 sec latency 198.192 ms 2024-09-19T06:34:57.032 INFO:tasks.workunit.client.1.smithi155.stdout: 10 37856 7.00 MB/sec execute 163 sec latency 130.939 ms 2024-09-19T06:34:57.851 INFO:tasks.workunit.client.0.smithi053.stdout: 10 145210 26.99 MB/sec execute 161 sec latency 250.788 ms 2024-09-19T06:34:58.032 INFO:tasks.workunit.client.1.smithi155.stdout: 10 38036 7.04 MB/sec execute 164 sec latency 85.819 ms 2024-09-19T06:34:58.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:58 smithi155 ceph-mon[35292]: pgmap v764: 129 pgs: 129 active+clean; 416 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 6.9 KiB/s rd, 2.2 MiB/s wr, 172 op/s 2024-09-19T06:34:58.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:58 smithi053 ceph-mon[27968]: pgmap v764: 129 pgs: 129 active+clean; 416 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 6.9 KiB/s rd, 2.2 MiB/s wr, 172 op/s 2024-09-19T06:34:58.851 INFO:tasks.workunit.client.0.smithi053.stdout: 10 146196 27.14 MB/sec execute 162 sec latency 109.125 ms 2024-09-19T06:34:59.032 INFO:tasks.workunit.client.1.smithi155.stdout: 10 38233 7.13 MB/sec execute 165 sec latency 86.281 ms 2024-09-19T06:34:59.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:34:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:34:59.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:34:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:34:59.851 INFO:tasks.workunit.client.0.smithi053.stdout: 10 146266 26.98 MB/sec execute 163 sec latency 388.491 ms 2024-09-19T06:35:00.032 INFO:tasks.workunit.client.1.smithi155.stdout: 10 38392 7.14 MB/sec execute 166 sec latency 246.723 ms 2024-09-19T06:35:00.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:00 smithi155 ceph-mon[35292]: pgmap v765: 129 pgs: 129 active+clean; 422 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 8.3 KiB/s rd, 3.0 MiB/s wr, 206 op/s 2024-09-19T06:35:00.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:00 smithi053 ceph-mon[27968]: pgmap v765: 129 pgs: 129 active+clean; 422 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 8.3 KiB/s rd, 3.0 MiB/s wr, 206 op/s 2024-09-19T06:35:00.851 INFO:tasks.workunit.client.0.smithi053.stdout: 10 146299 26.82 MB/sec execute 164 sec latency 395.226 ms 2024-09-19T06:35:01.032 INFO:tasks.workunit.client.1.smithi155.stdout: 10 38598 7.21 MB/sec execute 167 sec latency 180.370 ms 2024-09-19T06:35:01.852 INFO:tasks.workunit.client.0.smithi053.stdout: 10 146360 26.67 MB/sec execute 165 sec latency 491.492 ms 2024-09-19T06:35:02.032 INFO:tasks.workunit.client.1.smithi155.stdout: 10 38677 7.21 MB/sec execute 168 sec latency 201.024 ms 2024-09-19T06:35:02.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:02 smithi155 ceph-mon[35292]: pgmap v766: 129 pgs: 129 active+clean; 424 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 2.4 MiB/s wr, 138 op/s 2024-09-19T06:35:02.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:02 smithi053 ceph-mon[27968]: pgmap v766: 129 pgs: 129 active+clean; 424 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 2.4 MiB/s wr, 138 op/s 2024-09-19T06:35:02.851 INFO:tasks.workunit.client.0.smithi053.stdout: 10 146474 26.54 MB/sec execute 166 sec latency 375.742 ms 2024-09-19T06:35:03.032 INFO:tasks.workunit.client.1.smithi155.stdout: 10 38799 7.21 MB/sec execute 169 sec latency 197.969 ms 2024-09-19T06:35:03.852 INFO:tasks.workunit.client.0.smithi053.stdout: 10 147007 26.61 MB/sec execute 167 sec latency 384.401 ms 2024-09-19T06:35:04.032 INFO:tasks.workunit.client.1.smithi155.stdout: 10 38842 7.19 MB/sec execute 170 sec latency 227.081 ms 2024-09-19T06:35:04.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:04 smithi155 ceph-mon[35292]: pgmap v767: 129 pgs: 129 active+clean; 439 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 5.2 KiB/s rd, 3.6 MiB/s wr, 167 op/s 2024-09-19T06:35:04.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:04 smithi053 ceph-mon[27968]: pgmap v767: 129 pgs: 129 active+clean; 439 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 5.2 KiB/s rd, 3.6 MiB/s wr, 167 op/s 2024-09-19T06:35:04.852 INFO:tasks.workunit.client.0.smithi053.stdout: 10 147397 26.48 MB/sec execute 168 sec latency 555.194 ms 2024-09-19T06:35:05.032 INFO:tasks.workunit.client.1.smithi155.stdout: 10 38912 7.17 MB/sec execute 171 sec latency 227.025 ms 2024-09-19T06:35:05.852 INFO:tasks.workunit.client.0.smithi053.stdout: 10 148718 26.82 MB/sec execute 169 sec latency 117.565 ms 2024-09-19T06:35:06.032 INFO:tasks.workunit.client.1.smithi155.stdout: 10 39055 7.21 MB/sec execute 172 sec latency 96.975 ms 2024-09-19T06:35:06.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:06 smithi155 ceph-mon[35292]: pgmap v768: 129 pgs: 129 active+clean; 442 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.4 KiB/s rd, 3.3 MiB/s wr, 115 op/s 2024-09-19T06:35:06.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:06 smithi053 ceph-mon[27968]: pgmap v768: 129 pgs: 129 active+clean; 442 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.4 KiB/s rd, 3.3 MiB/s wr, 115 op/s 2024-09-19T06:35:06.852 INFO:tasks.workunit.client.0.smithi053.stdout: 10 149735 26.99 MB/sec execute 170 sec latency 152.287 ms 2024-09-19T06:35:07.032 INFO:tasks.workunit.client.1.smithi155.stdout: 10 39228 7.27 MB/sec execute 173 sec latency 120.406 ms 2024-09-19T06:35:07.852 INFO:tasks.workunit.client.0.smithi053.stdout: 10 149916 26.86 MB/sec execute 171 sec latency 95.702 ms 2024-09-19T06:35:08.032 INFO:tasks.workunit.client.1.smithi155.stdout: 10 39357 7.26 MB/sec execute 174 sec latency 108.725 ms 2024-09-19T06:35:08.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:08 smithi053 ceph-mon[27968]: pgmap v769: 129 pgs: 129 active+clean; 442 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.4 KiB/s rd, 3.7 MiB/s wr, 114 op/s 2024-09-19T06:35:08.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:08 smithi155 ceph-mon[35292]: pgmap v769: 129 pgs: 129 active+clean; 442 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.4 KiB/s rd, 3.7 MiB/s wr, 114 op/s 2024-09-19T06:35:08.852 INFO:tasks.workunit.client.0.smithi053.stdout: 10 151176 27.02 MB/sec execute 172 sec latency 136.836 ms 2024-09-19T06:35:09.032 INFO:tasks.workunit.client.1.smithi155.stdout: 10 39465 7.23 MB/sec execute 175 sec latency 84.481 ms 2024-09-19T06:35:09.852 INFO:tasks.workunit.client.0.smithi053.stdout: 10 152041 27.24 MB/sec execute 173 sec latency 304.892 ms 2024-09-19T06:35:10.032 INFO:tasks.workunit.client.1.smithi155.stdout: 10 39545 7.20 MB/sec execute 176 sec latency 195.137 ms 2024-09-19T06:35:10.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:10 smithi155 ceph-mon[35292]: pgmap v770: 129 pgs: 129 active+clean; 447 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 4.7 KiB/s rd, 4.5 MiB/s wr, 139 op/s 2024-09-19T06:35:10.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:10 smithi053 ceph-mon[27968]: pgmap v770: 129 pgs: 129 active+clean; 447 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 4.7 KiB/s rd, 4.5 MiB/s wr, 139 op/s 2024-09-19T06:35:10.852 INFO:tasks.workunit.client.0.smithi053.stdout: 10 152272 27.16 MB/sec execute 174 sec latency 400.670 ms 2024-09-19T06:35:11.032 INFO:tasks.workunit.client.1.smithi155.stdout: 10 39628 7.16 MB/sec execute 177 sec latency 201.521 ms 2024-09-19T06:35:11.852 INFO:tasks.workunit.client.0.smithi053.stdout: 10 152512 27.08 MB/sec execute 175 sec latency 459.584 ms 2024-09-19T06:35:12.032 INFO:tasks.workunit.client.1.smithi155.stdout: 10 39729 7.13 MB/sec execute 178 sec latency 161.152 ms 2024-09-19T06:35:12.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:12 smithi155 ceph-mon[35292]: pgmap v771: 129 pgs: 129 active+clean; 448 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.3 KiB/s rd, 4.0 MiB/s wr, 96 op/s 2024-09-19T06:35:12.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:12 smithi053 ceph-mon[27968]: pgmap v771: 129 pgs: 129 active+clean; 448 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.3 KiB/s rd, 4.0 MiB/s wr, 96 op/s 2024-09-19T06:35:12.852 INFO:tasks.workunit.client.0.smithi053.stdout: 10 153229 27.15 MB/sec execute 176 sec latency 498.745 ms 2024-09-19T06:35:13.032 INFO:tasks.workunit.client.1.smithi155.stdout: 10 39866 7.10 MB/sec execute 179 sec latency 149.800 ms 2024-09-19T06:35:13.852 INFO:tasks.workunit.client.0.smithi053.stdout: 10 153239 26.99 MB/sec execute 177 sec latency 408.907 ms 2024-09-19T06:35:14.032 INFO:tasks.workunit.client.1.smithi155.stdout: 10 39948 7.08 MB/sec execute 180 sec latency 193.055 ms 2024-09-19T06:35:14.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:14 smithi155 ceph-mon[35292]: pgmap v772: 129 pgs: 129 active+clean; 448 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 8.4 KiB/s rd, 4.4 MiB/s wr, 140 op/s 2024-09-19T06:35:14.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:14 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:35:14.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:14 smithi053 ceph-mon[27968]: pgmap v772: 129 pgs: 129 active+clean; 448 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 8.4 KiB/s rd, 4.4 MiB/s wr, 140 op/s 2024-09-19T06:35:14.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:14 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:35:14.852 INFO:tasks.workunit.client.0.smithi053.stdout: 10 153260 26.84 MB/sec execute 178 sec latency 450.684 ms 2024-09-19T06:35:15.032 INFO:tasks.workunit.client.1.smithi155.stdout: 10 40066 7.09 MB/sec execute 181 sec latency 205.921 ms 2024-09-19T06:35:15.852 INFO:tasks.workunit.client.0.smithi053.stdout: 10 153289 26.70 MB/sec execute 179 sec latency 447.105 ms 2024-09-19T06:35:16.032 INFO:tasks.workunit.client.1.smithi155.stdout: 10 40199 7.06 MB/sec execute 182 sec latency 132.665 ms 2024-09-19T06:35:16.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:16 smithi155 ceph-mon[35292]: pgmap v773: 129 pgs: 129 active+clean; 444 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 6.4 KiB/s rd, 3.0 MiB/s wr, 110 op/s 2024-09-19T06:35:16.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:16 smithi053 ceph-mon[27968]: pgmap v773: 129 pgs: 129 active+clean; 444 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 6.4 KiB/s rd, 3.0 MiB/s wr, 110 op/s 2024-09-19T06:35:16.852 INFO:tasks.workunit.client.0.smithi053.stdout: 10 153442 26.57 MB/sec execute 180 sec latency 264.837 ms 2024-09-19T06:35:17.032 INFO:tasks.workunit.client.1.smithi155.stdout: 10 40391 7.13 MB/sec execute 183 sec latency 90.115 ms 2024-09-19T06:35:17.852 INFO:tasks.workunit.client.0.smithi053.stdout: 10 154487 26.69 MB/sec execute 181 sec latency 166.291 ms 2024-09-19T06:35:18.032 INFO:tasks.workunit.client.1.smithi155.stdout: 10 40569 7.13 MB/sec execute 184 sec latency 93.746 ms 2024-09-19T06:35:18.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:18 smithi155 ceph-mon[35292]: pgmap v774: 129 pgs: 129 active+clean; 437 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 6.4 KiB/s rd, 3.2 MiB/s wr, 127 op/s 2024-09-19T06:35:18.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:18 smithi053 ceph-mon[27968]: pgmap v774: 129 pgs: 129 active+clean; 437 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 6.4 KiB/s rd, 3.2 MiB/s wr, 127 op/s 2024-09-19T06:35:18.852 INFO:tasks.workunit.client.0.smithi053.stdout: 10 155645 26.96 MB/sec execute 182 sec latency 145.775 ms 2024-09-19T06:35:19.032 INFO:tasks.workunit.client.1.smithi155.stdout: 10 40755 7.13 MB/sec execute 185 sec latency 110.623 ms 2024-09-19T06:35:19.852 INFO:tasks.workunit.client.0.smithi053.stdout: 10 156797 27.16 MB/sec execute 183 sec latency 122.038 ms 2024-09-19T06:35:20.033 INFO:tasks.workunit.client.1.smithi155.stdout: 10 40936 7.10 MB/sec execute 186 sec latency 72.279 ms 2024-09-19T06:35:20.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:20 smithi155 ceph-mon[35292]: pgmap v775: 129 pgs: 129 active+clean; 431 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 9.2 KiB/s rd, 3.4 MiB/s wr, 178 op/s 2024-09-19T06:35:20.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:20 smithi053 ceph-mon[27968]: pgmap v775: 129 pgs: 129 active+clean; 431 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 9.2 KiB/s rd, 3.4 MiB/s wr, 178 op/s 2024-09-19T06:35:20.852 INFO:tasks.workunit.client.0.smithi053.stdout: 10 156937 27.03 MB/sec execute 184 sec latency 100.629 ms 2024-09-19T06:35:21.033 INFO:tasks.workunit.client.1.smithi155.stdout: 10 41122 7.08 MB/sec execute 187 sec latency 81.268 ms 2024-09-19T06:35:21.852 INFO:tasks.workunit.client.0.smithi053.stdout: 10 158030 27.15 MB/sec execute 185 sec latency 146.083 ms 2024-09-19T06:35:22.033 INFO:tasks.workunit.client.1.smithi155.stdout: 10 41312 7.08 MB/sec execute 188 sec latency 74.234 ms 2024-09-19T06:35:22.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:22 smithi155 ceph-mon[35292]: pgmap v776: 129 pgs: 129 active+clean; 426 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 7.8 KiB/s rd, 2.6 MiB/s wr, 150 op/s 2024-09-19T06:35:22.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:22 smithi053 ceph-mon[27968]: pgmap v776: 129 pgs: 129 active+clean; 426 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 7.8 KiB/s rd, 2.6 MiB/s wr, 150 op/s 2024-09-19T06:35:22.852 INFO:tasks.workunit.client.0.smithi053.stdout: 10 159268 27.43 MB/sec execute 186 sec latency 131.844 ms 2024-09-19T06:35:23.033 INFO:tasks.workunit.client.1.smithi155.stdout: 10 41455 7.08 MB/sec execute 189 sec latency 108.047 ms 2024-09-19T06:35:23.852 INFO:tasks.workunit.client.0.smithi053.stdout: 10 160373 27.60 MB/sec execute 187 sec latency 101.615 ms 2024-09-19T06:35:24.033 INFO:tasks.workunit.client.1.smithi155.stdout: 10 41624 7.10 MB/sec execute 190 sec latency 88.443 ms 2024-09-19T06:35:24.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:24 smithi155 ceph-mon[35292]: pgmap v777: 129 pgs: 129 active+clean; 410 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 9.6 KiB/s rd, 2.8 MiB/s wr, 212 op/s 2024-09-19T06:35:24.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:24 smithi053 ceph-mon[27968]: pgmap v777: 129 pgs: 129 active+clean; 410 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 9.6 KiB/s rd, 2.8 MiB/s wr, 212 op/s 2024-09-19T06:35:24.852 INFO:tasks.workunit.client.0.smithi053.stdout: 10 160547 27.48 MB/sec execute 188 sec latency 94.075 ms 2024-09-19T06:35:25.033 INFO:tasks.workunit.client.1.smithi155.stdout: 10 41839 7.17 MB/sec execute 191 sec latency 97.224 ms 2024-09-19T06:35:25.852 INFO:tasks.workunit.client.0.smithi053.stdout: 10 161866 27.64 MB/sec execute 189 sec latency 116.975 ms 2024-09-19T06:35:26.033 INFO:tasks.workunit.client.1.smithi155.stdout: 10 42074 7.22 MB/sec execute 192 sec latency 109.346 ms 2024-09-19T06:35:26.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:26 smithi155 ceph-mon[35292]: pgmap v778: 129 pgs: 129 active+clean; 413 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 4.5 KiB/s rd, 2.6 MiB/s wr, 174 op/s 2024-09-19T06:35:26.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:26 smithi053 ceph-mon[27968]: pgmap v778: 129 pgs: 129 active+clean; 413 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 4.5 KiB/s rd, 2.6 MiB/s wr, 174 op/s 2024-09-19T06:35:26.853 INFO:tasks.workunit.client.0.smithi053.stdout: 10 162738 27.84 MB/sec execute 190 sec latency 311.484 ms 2024-09-19T06:35:27.033 INFO:tasks.workunit.client.1.smithi155.stdout: 10 42251 7.27 MB/sec execute 193 sec latency 80.091 ms 2024-09-19T06:35:27.853 INFO:tasks.workunit.client.0.smithi053.stdout: 10 163878 28.02 MB/sec execute 191 sec latency 202.334 ms 2024-09-19T06:35:28.033 INFO:tasks.workunit.client.1.smithi155.stdout: 10 42384 7.28 MB/sec execute 194 sec latency 121.975 ms 2024-09-19T06:35:28.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:28 smithi155 ceph-mon[35292]: pgmap v779: 129 pgs: 129 active+clean; 414 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 4.5 KiB/s rd, 2.9 MiB/s wr, 175 op/s 2024-09-19T06:35:28.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:28 smithi053 ceph-mon[27968]: pgmap v779: 129 pgs: 129 active+clean; 414 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 4.5 KiB/s rd, 2.9 MiB/s wr, 175 op/s 2024-09-19T06:35:28.853 INFO:tasks.workunit.client.0.smithi053.stdout: 10 164008 27.89 MB/sec execute 192 sec latency 225.822 ms 2024-09-19T06:35:29.033 INFO:tasks.workunit.client.1.smithi155.stdout: 10 42449 7.27 MB/sec execute 195 sec latency 106.937 ms 2024-09-19T06:35:29.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:29 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:35:29.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:35:29.853 INFO:tasks.workunit.client.0.smithi053.stdout: 10 165119 28.00 MB/sec execute 193 sec latency 129.092 ms 2024-09-19T06:35:30.033 INFO:tasks.workunit.client.1.smithi155.stdout: 10 42589 7.31 MB/sec execute 196 sec latency 93.441 ms 2024-09-19T06:35:30.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:30 smithi155 ceph-mon[35292]: pgmap v780: 129 pgs: 129 active+clean; 430 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 6.7 KiB/s rd, 4.1 MiB/s wr, 195 op/s 2024-09-19T06:35:30.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:30 smithi053 ceph-mon[27968]: pgmap v780: 129 pgs: 129 active+clean; 430 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 6.7 KiB/s rd, 4.1 MiB/s wr, 195 op/s 2024-09-19T06:35:30.853 INFO:tasks.workunit.client.0.smithi053.stdout: 10 166142 28.22 MB/sec execute 194 sec latency 323.449 ms 2024-09-19T06:35:31.033 INFO:tasks.workunit.client.1.smithi155.stdout: 10 42726 7.33 MB/sec execute 197 sec latency 160.699 ms 2024-09-19T06:35:31.853 INFO:tasks.workunit.client.0.smithi053.stdout: 10 167421 28.42 MB/sec execute 195 sec latency 354.567 ms 2024-09-19T06:35:32.033 INFO:tasks.workunit.client.1.smithi155.stdout: 10 42852 7.32 MB/sec execute 198 sec latency 100.041 ms 2024-09-19T06:35:32.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:32 smithi155 ceph-mon[35292]: pgmap v781: 129 pgs: 129 active+clean; 432 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 4.0 KiB/s rd, 3.6 MiB/s wr, 141 op/s 2024-09-19T06:35:32.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:32 smithi053 ceph-mon[27968]: pgmap v781: 129 pgs: 129 active+clean; 432 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 4.0 KiB/s rd, 3.6 MiB/s wr, 141 op/s 2024-09-19T06:35:32.853 INFO:tasks.workunit.client.0.smithi053.stdout: 10 167557 28.30 MB/sec execute 196 sec latency 87.176 ms 2024-09-19T06:35:33.033 INFO:tasks.workunit.client.1.smithi155.stdout: 10 42981 7.32 MB/sec execute 199 sec latency 127.288 ms 2024-09-19T06:35:33.853 INFO:tasks.workunit.client.0.smithi053.stdout: 10 168716 28.41 MB/sec execute 197 sec latency 116.117 ms 2024-09-19T06:35:34.033 INFO:tasks.workunit.client.1.smithi155.stdout: 10 43084 7.29 MB/sec execute 200 sec latency 114.580 ms 2024-09-19T06:35:34.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:34 smithi053 ceph-mon[27968]: pgmap v782: 129 pgs: 129 active+clean; 438 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 6.0 KiB/s rd, 4.8 MiB/s wr, 174 op/s 2024-09-19T06:35:34.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:34 smithi155 ceph-mon[35292]: pgmap v782: 129 pgs: 129 active+clean; 438 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 6.0 KiB/s rd, 4.8 MiB/s wr, 174 op/s 2024-09-19T06:35:34.853 INFO:tasks.workunit.client.0.smithi053.stdout: 10 169828 28.64 MB/sec execute 198 sec latency 126.514 ms 2024-09-19T06:35:35.033 INFO:tasks.workunit.client.1.smithi155.stdout: 10 43190 7.26 MB/sec execute 201 sec latency 127.186 ms 2024-09-19T06:35:35.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:35 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:35:35.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:35 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:35:35.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:35 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:35:35.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:35 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:35:35.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:35 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:35:35.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:35 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:35:35.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:35 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:35:35.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:35 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:35:35.853 INFO:tasks.workunit.client.0.smithi053.stdout: 10 171008 28.82 MB/sec execute 199 sec latency 94.906 ms 2024-09-19T06:35:36.033 INFO:tasks.workunit.client.1.smithi155.stdout: 10 43320 7.23 MB/sec execute 202 sec latency 113.336 ms 2024-09-19T06:35:36.853 INFO:tasks.workunit.client.0.smithi053.stdout: 10 171074 28.68 MB/sec execute 200 sec latency 389.138 ms 2024-09-19T06:35:37.033 INFO:tasks.workunit.client.1.smithi155.stdout: 10 43418 7.21 MB/sec execute 203 sec latency 175.277 ms 2024-09-19T06:35:37.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:36 smithi053 ceph-mon[27968]: pgmap v783: 129 pgs: 129 active+clean; 440 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 4.2 KiB/s rd, 4.7 MiB/s wr, 114 op/s 2024-09-19T06:35:37.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:36 smithi155 ceph-mon[35292]: pgmap v783: 129 pgs: 129 active+clean; 440 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 4.2 KiB/s rd, 4.7 MiB/s wr, 114 op/s 2024-09-19T06:35:37.853 INFO:tasks.workunit.client.0.smithi053.stdout: 10 171079 28.54 MB/sec execute 201 sec latency 452.511 ms 2024-09-19T06:35:38.033 INFO:tasks.workunit.client.1.smithi155.stdout: 10 43539 7.19 MB/sec execute 204 sec latency 168.919 ms 2024-09-19T06:35:38.853 INFO:tasks.workunit.client.0.smithi053.stdout: 10 171110 28.40 MB/sec execute 202 sec latency 400.641 ms 2024-09-19T06:35:39.033 INFO:tasks.workunit.client.1.smithi155.stdout: 10 43648 7.19 MB/sec execute 205 sec latency 199.502 ms 2024-09-19T06:35:39.034 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:38 smithi155 ceph-mon[35292]: pgmap v784: 129 pgs: 129 active+clean; 441 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 4.2 KiB/s rd, 4.5 MiB/s wr, 108 op/s 2024-09-19T06:35:39.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:38 smithi053 ceph-mon[27968]: pgmap v784: 129 pgs: 129 active+clean; 441 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 4.2 KiB/s rd, 4.5 MiB/s wr, 108 op/s 2024-09-19T06:35:39.853 INFO:tasks.workunit.client.0.smithi053.stdout: 10 171180 28.27 MB/sec execute 203 sec latency 417.930 ms 2024-09-19T06:35:40.033 INFO:tasks.workunit.client.1.smithi155.stdout: 10 43769 7.19 MB/sec execute 206 sec latency 160.874 ms 2024-09-19T06:35:40.853 INFO:tasks.workunit.client.0.smithi053.stdout: 10 171276 28.16 MB/sec execute 204 sec latency 451.884 ms 2024-09-19T06:35:41.033 INFO:tasks.workunit.client.1.smithi155.stdout: 10 43888 7.21 MB/sec execute 207 sec latency 194.000 ms 2024-09-19T06:35:41.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:40 smithi053 ceph-mon[27968]: pgmap v785: 129 pgs: 129 active+clean; 432 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 9.7 KiB/s rd, 4.7 MiB/s wr, 155 op/s 2024-09-19T06:35:41.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:40 smithi155 ceph-mon[35292]: pgmap v785: 129 pgs: 129 active+clean; 432 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 9.7 KiB/s rd, 4.7 MiB/s wr, 155 op/s 2024-09-19T06:35:41.853 INFO:tasks.workunit.client.0.smithi053.stdout: 10 171816 28.21 MB/sec execute 205 sec latency 520.716 ms 2024-09-19T06:35:42.033 INFO:tasks.workunit.client.1.smithi155.stdout: 10 44013 7.19 MB/sec execute 208 sec latency 176.924 ms 2024-09-19T06:35:42.856 INFO:tasks.workunit.client.0.smithi053.stdout: 10 172013 28.08 MB/sec execute 206 sec latency 524.250 ms 2024-09-19T06:35:43.033 INFO:tasks.workunit.client.1.smithi155.stdout: 10 44121 7.19 MB/sec execute 209 sec latency 153.163 ms 2024-09-19T06:35:43.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:42 smithi053 ceph-mon[27968]: pgmap v786: 129 pgs: 129 active+clean; 428 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 7.5 KiB/s rd, 3.2 MiB/s wr, 123 op/s 2024-09-19T06:35:43.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:42 smithi155 ceph-mon[35292]: pgmap v786: 129 pgs: 129 active+clean; 428 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 7.5 KiB/s rd, 3.2 MiB/s wr, 123 op/s 2024-09-19T06:35:43.853 INFO:tasks.workunit.client.0.smithi053.stdout: 10 172262 27.97 MB/sec execute 207 sec latency 543.326 ms 2024-09-19T06:35:44.033 INFO:tasks.workunit.client.1.smithi155.stdout: 10 44264 7.19 MB/sec execute 210 sec latency 148.808 ms 2024-09-19T06:35:44.078 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:43 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:35:44.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:43 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:35:44.853 INFO:tasks.workunit.client.0.smithi053.stdout: 10 172708 27.92 MB/sec execute 208 sec latency 417.618 ms 2024-09-19T06:35:45.033 INFO:tasks.workunit.client.1.smithi155.stdout: 10 44387 7.16 MB/sec execute 211 sec latency 174.305 ms 2024-09-19T06:35:45.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:44 smithi053 ceph-mon[27968]: pgmap v787: 129 pgs: 129 active+clean; 423 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 9.7 KiB/s rd, 3.8 MiB/s wr, 180 op/s 2024-09-19T06:35:45.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:44 smithi155 ceph-mon[35292]: pgmap v787: 129 pgs: 129 active+clean; 423 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 9.7 KiB/s rd, 3.8 MiB/s wr, 180 op/s 2024-09-19T06:35:45.853 INFO:tasks.workunit.client.0.smithi053.stdout: 10 172910 27.87 MB/sec execute 209 sec latency 337.993 ms 2024-09-19T06:35:46.033 INFO:tasks.workunit.client.1.smithi155.stdout: 10 44515 7.14 MB/sec execute 212 sec latency 160.046 ms 2024-09-19T06:35:46.853 INFO:tasks.workunit.client.0.smithi053.stdout: 10 173310 27.90 MB/sec execute 210 sec latency 377.923 ms 2024-09-19T06:35:47.034 INFO:tasks.workunit.client.1.smithi155.stdout: 10 44633 7.11 MB/sec execute 213 sec latency 154.013 ms 2024-09-19T06:35:47.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:46 smithi053 ceph-mon[27968]: pgmap v788: 129 pgs: 129 active+clean; 421 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 7.7 KiB/s rd, 2.6 MiB/s wr, 150 op/s 2024-09-19T06:35:47.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:46 smithi155 ceph-mon[35292]: pgmap v788: 129 pgs: 129 active+clean; 421 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 7.7 KiB/s rd, 2.6 MiB/s wr, 150 op/s 2024-09-19T06:35:47.853 INFO:tasks.workunit.client.0.smithi053.stdout: 10 173515 27.82 MB/sec execute 211 sec latency 367.247 ms 2024-09-19T06:35:48.034 INFO:tasks.workunit.client.1.smithi155.stdout: 10 44742 7.09 MB/sec execute 214 sec latency 182.339 ms 2024-09-19T06:35:48.853 INFO:tasks.workunit.client.0.smithi053.stdout: 10 173766 27.76 MB/sec execute 212 sec latency 475.905 ms 2024-09-19T06:35:49.034 INFO:tasks.workunit.client.1.smithi155.stdout: 10 44885 7.08 MB/sec execute 215 sec latency 189.006 ms 2024-09-19T06:35:49.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:48 smithi053 ceph-mon[27968]: pgmap v789: 129 pgs: 129 active+clean; 421 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 7.7 KiB/s rd, 2.1 MiB/s wr, 154 op/s 2024-09-19T06:35:49.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:48 smithi155 ceph-mon[35292]: pgmap v789: 129 pgs: 129 active+clean; 421 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 7.7 KiB/s rd, 2.1 MiB/s wr, 154 op/s 2024-09-19T06:35:49.853 INFO:tasks.workunit.client.0.smithi053.stdout: 10 173967 27.68 MB/sec execute 213 sec latency 569.170 ms 2024-09-19T06:35:50.034 INFO:tasks.workunit.client.1.smithi155.stdout: 10 45014 7.09 MB/sec execute 216 sec latency 160.746 ms 2024-09-19T06:35:50.854 INFO:tasks.workunit.client.0.smithi053.stdout: 10 174505 27.69 MB/sec execute 214 sec latency 500.671 ms 2024-09-19T06:35:51.034 INFO:tasks.workunit.client.1.smithi155.stdout: 10 45147 7.10 MB/sec execute 217 sec latency 160.592 ms 2024-09-19T06:35:51.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:50 smithi053 ceph-mon[27968]: pgmap v790: 129 pgs: 129 active+clean; 424 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 9.0 KiB/s rd, 2.2 MiB/s wr, 189 op/s 2024-09-19T06:35:51.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:50 smithi155 ceph-mon[35292]: pgmap v790: 129 pgs: 129 active+clean; 424 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 9.0 KiB/s rd, 2.2 MiB/s wr, 189 op/s 2024-09-19T06:35:51.853 INFO:tasks.workunit.client.0.smithi053.stdout: 10 174518 27.56 MB/sec execute 215 sec latency 433.925 ms 2024-09-19T06:35:52.034 INFO:tasks.workunit.client.1.smithi155.stdout: 10 45267 7.11 MB/sec execute 218 sec latency 159.756 ms 2024-09-19T06:35:52.854 INFO:tasks.workunit.client.0.smithi053.stdout: 10 174557 27.44 MB/sec execute 216 sec latency 427.035 ms 2024-09-19T06:35:53.034 INFO:tasks.workunit.client.1.smithi155.stdout: 10 45395 7.13 MB/sec execute 219 sec latency 174.414 ms 2024-09-19T06:35:53.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:52 smithi053 ceph-mon[27968]: pgmap v791: 129 pgs: 129 active+clean; 424 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.7 MiB/s wr, 137 op/s 2024-09-19T06:35:53.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:52 smithi155 ceph-mon[35292]: pgmap v791: 129 pgs: 129 active+clean; 424 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.7 MiB/s wr, 137 op/s 2024-09-19T06:35:53.854 INFO:tasks.workunit.client.0.smithi053.stdout: 10 174574 27.31 MB/sec execute 217 sec latency 475.464 ms 2024-09-19T06:35:54.034 INFO:tasks.workunit.client.1.smithi155.stdout: 10 45535 7.16 MB/sec execute 220 sec latency 178.307 ms 2024-09-19T06:35:54.854 INFO:tasks.workunit.client.0.smithi053.stdout: 10 174760 27.21 MB/sec execute 218 sec latency 283.335 ms 2024-09-19T06:35:55.034 INFO:tasks.workunit.client.1.smithi155.stdout: 10 45768 7.21 MB/sec execute 221 sec latency 110.561 ms 2024-09-19T06:35:55.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:54 smithi053 ceph-mon[27968]: pgmap v792: 129 pgs: 129 active+clean; 433 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 4.6 KiB/s rd, 2.8 MiB/s wr, 172 op/s 2024-09-19T06:35:55.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:54 smithi155 ceph-mon[35292]: pgmap v792: 129 pgs: 129 active+clean; 433 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 4.6 KiB/s rd, 2.8 MiB/s wr, 172 op/s 2024-09-19T06:35:55.854 INFO:tasks.workunit.client.0.smithi053.stdout: 10 176237 27.39 MB/sec execute 219 sec latency 157.857 ms 2024-09-19T06:35:56.034 INFO:tasks.workunit.client.1.smithi155.stdout: 10 45921 7.23 MB/sec execute 222 sec latency 134.030 ms 2024-09-19T06:35:56.854 INFO:tasks.workunit.client.0.smithi053.stdout: 10 177218 27.60 MB/sec execute 220 sec latency 134.583 ms 2024-09-19T06:35:57.034 INFO:tasks.workunit.client.1.smithi155.stdout: 10 45985 7.21 MB/sec execute 223 sec latency 96.668 ms 2024-09-19T06:35:57.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:56 smithi053 ceph-mon[27968]: pgmap v793: 129 pgs: 129 active+clean; 439 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 2.4 KiB/s rd, 2.7 MiB/s wr, 117 op/s 2024-09-19T06:35:57.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:56 smithi155 ceph-mon[35292]: pgmap v793: 129 pgs: 129 active+clean; 439 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 2.4 KiB/s rd, 2.7 MiB/s wr, 117 op/s 2024-09-19T06:35:57.854 INFO:tasks.workunit.client.0.smithi053.stdout: 10 178110 27.68 MB/sec execute 221 sec latency 111.982 ms 2024-09-19T06:35:58.035 INFO:tasks.workunit.client.1.smithi155.stdout: 10 46111 7.24 MB/sec execute 224 sec latency 77.069 ms 2024-09-19T06:35:58.854 INFO:tasks.workunit.client.0.smithi053.stdout: 10 178279 27.58 MB/sec execute 222 sec latency 214.810 ms 2024-09-19T06:35:59.034 INFO:tasks.workunit.client.1.smithi155.stdout: 10 46265 7.25 MB/sec execute 225 sec latency 76.765 ms 2024-09-19T06:35:59.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:58 smithi053 ceph-mon[27968]: pgmap v794: 129 pgs: 129 active+clean; 442 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 2.4 KiB/s rd, 2.7 MiB/s wr, 114 op/s 2024-09-19T06:35:59.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:35:58 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:35:59.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:58 smithi155 ceph-mon[35292]: pgmap v794: 129 pgs: 129 active+clean; 442 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 2.4 KiB/s rd, 2.7 MiB/s wr, 114 op/s 2024-09-19T06:35:59.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:35:58 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:35:59.854 INFO:tasks.workunit.client.0.smithi053.stdout: 10 178964 27.65 MB/sec execute 223 sec latency 381.717 ms 2024-09-19T06:36:00.034 INFO:tasks.workunit.client.1.smithi155.stdout: 10 46388 7.26 MB/sec execute 226 sec latency 151.688 ms 2024-09-19T06:36:00.854 INFO:tasks.workunit.client.0.smithi053.stdout: 10 179152 27.54 MB/sec execute 224 sec latency 712.761 ms 2024-09-19T06:36:01.034 INFO:tasks.workunit.client.1.smithi155.stdout: 10 46468 7.24 MB/sec execute 227 sec latency 168.900 ms 2024-09-19T06:36:01.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:00 smithi053 ceph-mon[27968]: pgmap v795: 129 pgs: 129 active+clean; 449 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 4.2 KiB/s rd, 3.7 MiB/s wr, 134 op/s 2024-09-19T06:36:01.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:00 smithi155 ceph-mon[35292]: pgmap v795: 129 pgs: 129 active+clean; 449 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 4.2 KiB/s rd, 3.7 MiB/s wr, 134 op/s 2024-09-19T06:36:01.854 INFO:tasks.workunit.client.0.smithi053.stdout: 10 179448 27.44 MB/sec execute 225 sec latency 539.125 ms 2024-09-19T06:36:02.034 INFO:tasks.workunit.client.1.smithi155.stdout: 10 46576 7.23 MB/sec execute 228 sec latency 175.460 ms 2024-09-19T06:36:02.854 INFO:tasks.workunit.client.0.smithi053.stdout: 10 179805 27.40 MB/sec execute 226 sec latency 530.573 ms 2024-09-19T06:36:03.034 INFO:tasks.workunit.client.1.smithi155.stdout: 10 46646 7.20 MB/sec execute 229 sec latency 157.132 ms 2024-09-19T06:36:03.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:02 smithi053 ceph-mon[27968]: pgmap v796: 129 pgs: 129 active+clean; 450 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 2.9 KiB/s rd, 3.7 MiB/s wr, 94 op/s 2024-09-19T06:36:03.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:02 smithi155 ceph-mon[35292]: pgmap v796: 129 pgs: 129 active+clean; 450 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 2.9 KiB/s rd, 3.7 MiB/s wr, 94 op/s 2024-09-19T06:36:03.854 INFO:tasks.workunit.client.0.smithi053.stdout: 10 180025 27.36 MB/sec execute 227 sec latency 511.796 ms 2024-09-19T06:36:04.034 INFO:tasks.workunit.client.1.smithi155.stdout: 10 46729 7.17 MB/sec execute 230 sec latency 171.732 ms 2024-09-19T06:36:04.854 INFO:tasks.workunit.client.0.smithi053.stdout: 10 180349 27.37 MB/sec execute 228 sec latency 528.608 ms 2024-09-19T06:36:05.034 INFO:tasks.workunit.client.1.smithi155.stdout: 10 46803 7.15 MB/sec execute 231 sec latency 261.773 ms 2024-09-19T06:36:05.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:04 smithi053 ceph-mon[27968]: pgmap v797: 129 pgs: 129 active+clean; 454 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 5.3 KiB/s rd, 4.8 MiB/s wr, 130 op/s 2024-09-19T06:36:05.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:04 smithi155 ceph-mon[35292]: pgmap v797: 129 pgs: 129 active+clean; 454 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 5.3 KiB/s rd, 4.8 MiB/s wr, 130 op/s 2024-09-19T06:36:05.854 INFO:tasks.workunit.client.0.smithi053.stdout: 10 180488 27.28 MB/sec execute 229 sec latency 439.139 ms 2024-09-19T06:36:06.034 INFO:tasks.workunit.client.1.smithi155.stdout: 10 46903 7.13 MB/sec execute 232 sec latency 168.310 ms 2024-09-19T06:36:06.854 INFO:tasks.workunit.client.0.smithi053.stdout: 10 180857 27.25 MB/sec execute 230 sec latency 391.670 ms 2024-09-19T06:36:07.035 INFO:tasks.workunit.client.1.smithi155.stdout: 10 46997 7.11 MB/sec execute 233 sec latency 310.925 ms 2024-09-19T06:36:07.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:06 smithi053 ceph-mon[27968]: pgmap v798: 129 pgs: 129 active+clean; 454 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 4.2 KiB/s rd, 3.9 MiB/s wr, 89 op/s 2024-09-19T06:36:07.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:06 smithi155 ceph-mon[35292]: pgmap v798: 129 pgs: 129 active+clean; 454 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 4.2 KiB/s rd, 3.9 MiB/s wr, 89 op/s 2024-09-19T06:36:07.854 INFO:tasks.workunit.client.0.smithi053.stdout: 10 181057 27.18 MB/sec execute 231 sec latency 539.662 ms 2024-09-19T06:36:08.035 INFO:tasks.workunit.client.1.smithi155.stdout: 10 47125 7.10 MB/sec execute 234 sec latency 202.029 ms 2024-09-19T06:36:08.854 INFO:tasks.workunit.client.0.smithi053.stdout: 10 181598 27.19 MB/sec execute 232 sec latency 437.058 ms 2024-09-19T06:36:09.034 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:08 smithi155 ceph-mon[35292]: pgmap v799: 129 pgs: 129 active+clean; 454 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 4.2 KiB/s rd, 3.5 MiB/s wr, 114 op/s 2024-09-19T06:36:09.035 INFO:tasks.workunit.client.1.smithi155.stdout: 10 47229 7.09 MB/sec execute 235 sec latency 214.604 ms 2024-09-19T06:36:09.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:08 smithi053 ceph-mon[27968]: pgmap v799: 129 pgs: 129 active+clean; 454 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 4.2 KiB/s rd, 3.5 MiB/s wr, 114 op/s 2024-09-19T06:36:09.854 INFO:tasks.workunit.client.0.smithi053.stdout: 10 181617 27.07 MB/sec execute 233 sec latency 475.993 ms 2024-09-19T06:36:10.035 INFO:tasks.workunit.client.1.smithi155.stdout: 10 47354 7.09 MB/sec execute 236 sec latency 194.520 ms 2024-09-19T06:36:10.854 INFO:tasks.workunit.client.0.smithi053.stdout: 10 181677 26.96 MB/sec execute 234 sec latency 359.429 ms 2024-09-19T06:36:11.035 INFO:tasks.workunit.client.1.smithi155.stdout: 10 47527 7.12 MB/sec execute 237 sec latency 108.365 ms 2024-09-19T06:36:11.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:10 smithi053 ceph-mon[27968]: pgmap v800: 129 pgs: 129 active+clean; 445 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 8.3 KiB/s rd, 3.5 MiB/s wr, 131 op/s 2024-09-19T06:36:11.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:10 smithi155 ceph-mon[35292]: pgmap v800: 129 pgs: 129 active+clean; 445 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 8.3 KiB/s rd, 3.5 MiB/s wr, 131 op/s 2024-09-19T06:36:11.854 INFO:tasks.workunit.client.0.smithi053.stdout: 10 181912 26.88 MB/sec execute 235 sec latency 482.691 ms 2024-09-19T06:36:12.035 INFO:tasks.workunit.client.1.smithi155.stdout: 10 47722 7.14 MB/sec execute 238 sec latency 81.429 ms 2024-09-19T06:36:12.854 INFO:tasks.workunit.client.0.smithi053.stdout: 10 183328 27.03 MB/sec execute 236 sec latency 123.918 ms 2024-09-19T06:36:13.035 INFO:tasks.workunit.client.1.smithi155.stdout: 10 47887 7.12 MB/sec execute 239 sec latency 87.757 ms 2024-09-19T06:36:13.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:12 smithi053 ceph-mon[27968]: pgmap v801: 129 pgs: 129 active+clean; 442 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 6.5 KiB/s rd, 2.7 MiB/s wr, 123 op/s 2024-09-19T06:36:13.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:12 smithi155 ceph-mon[35292]: pgmap v801: 129 pgs: 129 active+clean; 442 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 6.5 KiB/s rd, 2.7 MiB/s wr, 123 op/s 2024-09-19T06:36:13.854 INFO:tasks.workunit.client.0.smithi053.stdout: 10 184405 27.25 MB/sec execute 237 sec latency 113.900 ms 2024-09-19T06:36:14.035 INFO:tasks.workunit.client.1.smithi155.stdout: 10 48065 7.10 MB/sec execute 240 sec latency 85.024 ms 2024-09-19T06:36:14.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:13 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:36:14.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:13 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:36:14.855 INFO:tasks.workunit.client.0.smithi053.stdout: 10 185206 27.31 MB/sec execute 238 sec latency 201.157 ms 2024-09-19T06:36:15.035 INFO:tasks.workunit.client.1.smithi155.stdout: 10 48227 7.09 MB/sec execute 241 sec latency 148.071 ms 2024-09-19T06:36:15.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:14 smithi155 ceph-mon[35292]: pgmap v802: 129 pgs: 129 active+clean; 436 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 8.9 KiB/s rd, 3.0 MiB/s wr, 161 op/s 2024-09-19T06:36:15.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:14 smithi053 ceph-mon[27968]: pgmap v802: 129 pgs: 129 active+clean; 436 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 8.9 KiB/s rd, 3.0 MiB/s wr, 161 op/s 2024-09-19T06:36:15.855 INFO:tasks.workunit.client.0.smithi053.stdout: 10 185370 27.21 MB/sec execute 239 sec latency 296.732 ms 2024-09-19T06:36:16.035 INFO:tasks.workunit.client.1.smithi155.stdout: 10 48381 7.07 MB/sec execute 242 sec latency 133.788 ms 2024-09-19T06:36:16.855 INFO:tasks.workunit.client.0.smithi053.stdout: 10 185522 27.14 MB/sec execute 240 sec latency 499.159 ms 2024-09-19T06:36:17.035 INFO:tasks.workunit.client.1.smithi155.stdout: 10 48535 7.09 MB/sec execute 243 sec latency 200.585 ms 2024-09-19T06:36:17.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:16 smithi155 ceph-mon[35292]: pgmap v803: 129 pgs: 129 active+clean; 438 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 6.5 KiB/s rd, 2.3 MiB/s wr, 166 op/s 2024-09-19T06:36:17.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:16 smithi053 ceph-mon[27968]: pgmap v803: 129 pgs: 129 active+clean; 438 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 6.5 KiB/s rd, 2.3 MiB/s wr, 166 op/s 2024-09-19T06:36:17.855 INFO:tasks.workunit.client.0.smithi053.stdout: 10 186054 27.17 MB/sec execute 241 sec latency 547.253 ms 2024-09-19T06:36:18.035 INFO:tasks.workunit.client.1.smithi155.stdout: 10 48656 7.09 MB/sec execute 244 sec latency 160.708 ms 2024-09-19T06:36:18.855 INFO:tasks.workunit.client.0.smithi053.stdout: 10 186242 27.07 MB/sec execute 242 sec latency 399.946 ms 2024-09-19T06:36:19.036 INFO:tasks.workunit.client.1.smithi155.stdout: 10 48760 7.09 MB/sec execute 245 sec latency 182.989 ms 2024-09-19T06:36:19.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:18 smithi155 ceph-mon[35292]: pgmap v804: 129 pgs: 129 active+clean; 440 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 6.5 KiB/s rd, 2.3 MiB/s wr, 182 op/s 2024-09-19T06:36:19.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:18 smithi053 ceph-mon[27968]: pgmap v804: 129 pgs: 129 active+clean; 440 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 6.5 KiB/s rd, 2.3 MiB/s wr, 182 op/s 2024-09-19T06:36:19.855 INFO:tasks.workunit.client.0.smithi053.stdout: 10 186554 26.99 MB/sec execute 243 sec latency 444.959 ms 2024-09-19T06:36:20.036 INFO:tasks.workunit.client.1.smithi155.stdout: 10 48854 7.09 MB/sec execute 246 sec latency 197.629 ms 2024-09-19T06:36:20.855 INFO:tasks.workunit.client.0.smithi053.stdout: 10 186940 26.95 MB/sec execute 244 sec latency 434.503 ms 2024-09-19T06:36:21.035 INFO:tasks.workunit.client.1.smithi155.stdout: 10 49057 7.15 MB/sec execute 247 sec latency 148.985 ms 2024-09-19T06:36:21.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:20 smithi155 ceph-mon[35292]: pgmap v805: 129 pgs: 129 active+clean; 442 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 8.2 KiB/s rd, 2.6 MiB/s wr, 176 op/s 2024-09-19T06:36:21.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:20 smithi053 ceph-mon[27968]: pgmap v805: 129 pgs: 129 active+clean; 442 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 8.2 KiB/s rd, 2.6 MiB/s wr, 176 op/s 2024-09-19T06:36:21.855 INFO:tasks.workunit.client.0.smithi053.stdout: 10 187243 26.96 MB/sec execute 245 sec latency 530.245 ms 2024-09-19T06:36:22.036 INFO:tasks.workunit.client.1.smithi155.stdout: 10 49222 7.17 MB/sec execute 248 sec latency 203.315 ms 2024-09-19T06:36:22.855 INFO:tasks.workunit.client.0.smithi053.stdout: 10 188684 27.19 MB/sec execute 246 sec latency 451.050 ms 2024-09-19T06:36:23.036 INFO:tasks.workunit.client.1.smithi155.stdout: 10 49407 7.20 MB/sec execute 249 sec latency 100.602 ms 2024-09-19T06:36:23.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:22 smithi053 ceph-mon[27968]: pgmap v806: 129 pgs: 129 active+clean; 445 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 4.2 KiB/s rd, 2.7 MiB/s wr, 160 op/s 2024-09-19T06:36:23.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:22 smithi155 ceph-mon[35292]: pgmap v806: 129 pgs: 129 active+clean; 445 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 4.2 KiB/s rd, 2.7 MiB/s wr, 160 op/s 2024-09-19T06:36:23.855 INFO:tasks.workunit.client.0.smithi053.stdout: 10 188810 27.10 MB/sec execute 247 sec latency 102.569 ms 2024-09-19T06:36:24.036 INFO:tasks.workunit.client.1.smithi155.stdout: 10 49507 7.19 MB/sec execute 250 sec latency 86.642 ms 2024-09-19T06:36:24.855 INFO:tasks.workunit.client.0.smithi053.stdout: 10 189665 27.18 MB/sec execute 248 sec latency 118.676 ms 2024-09-19T06:36:25.036 INFO:tasks.workunit.client.1.smithi155.stdout: 10 49576 7.17 MB/sec execute 251 sec latency 83.288 ms 2024-09-19T06:36:25.220 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:24 smithi053 ceph-mon[27968]: pgmap v807: 129 pgs: 129 active+clean; 433 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 6.0 KiB/s rd, 3.9 MiB/s wr, 177 op/s 2024-09-19T06:36:25.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:24 smithi155 ceph-mon[35292]: pgmap v807: 129 pgs: 129 active+clean; 433 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 6.0 KiB/s rd, 3.9 MiB/s wr, 177 op/s 2024-09-19T06:36:25.855 INFO:tasks.workunit.client.0.smithi053.stdout: 10 190868 27.33 MB/sec execute 249 sec latency 121.047 ms 2024-09-19T06:36:26.036 INFO:tasks.workunit.client.1.smithi155.stdout: 10 49712 7.20 MB/sec execute 252 sec latency 102.161 ms 2024-09-19T06:36:26.855 INFO:tasks.workunit.client.0.smithi053.stdout: 10 192228 27.52 MB/sec execute 250 sec latency 124.821 ms 2024-09-19T06:36:27.036 INFO:tasks.workunit.client.1.smithi155.stdout: 10 49894 7.24 MB/sec execute 253 sec latency 108.705 ms 2024-09-19T06:36:27.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:26 smithi155 ceph-mon[35292]: pgmap v808: 129 pgs: 129 active+clean; 437 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 3.6 KiB/s rd, 3.6 MiB/s wr, 141 op/s 2024-09-19T06:36:27.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:26 smithi053 ceph-mon[27968]: pgmap v808: 129 pgs: 129 active+clean; 437 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 3.6 KiB/s rd, 3.6 MiB/s wr, 141 op/s 2024-09-19T06:36:27.855 INFO:tasks.workunit.client.0.smithi053.stdout: 10 192361 27.43 MB/sec execute 251 sec latency 162.911 ms 2024-09-19T06:36:28.036 INFO:tasks.workunit.client.1.smithi155.stdout: 10 50025 7.23 MB/sec execute 254 sec latency 93.709 ms 2024-09-19T06:36:28.855 INFO:tasks.workunit.client.0.smithi053.stdout: 10 193236 27.51 MB/sec execute 252 sec latency 110.121 ms 2024-09-19T06:36:29.036 INFO:tasks.workunit.client.1.smithi155.stdout: 10 50136 7.21 MB/sec execute 255 sec latency 117.652 ms 2024-09-19T06:36:29.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:28 smithi155 ceph-mon[35292]: pgmap v809: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 3.6 KiB/s rd, 3.7 MiB/s wr, 112 op/s 2024-09-19T06:36:29.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:28 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:36:29.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:28 smithi053 ceph-mon[27968]: pgmap v809: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 3.6 KiB/s rd, 3.7 MiB/s wr, 112 op/s 2024-09-19T06:36:29.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:28 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:36:29.855 INFO:tasks.workunit.client.0.smithi053.stdout: 10 194483 27.68 MB/sec execute 253 sec latency 133.552 ms 2024-09-19T06:36:30.036 INFO:tasks.workunit.client.1.smithi155.stdout: 10 50234 7.19 MB/sec execute 256 sec latency 115.534 ms 2024-09-19T06:36:30.855 INFO:tasks.workunit.client.0.smithi053.stdout: 10 195778 27.84 MB/sec execute 254 sec latency 175.950 ms 2024-09-19T06:36:31.036 INFO:tasks.workunit.client.1.smithi155.stdout: 10 50355 7.17 MB/sec execute 257 sec latency 116.289 ms 2024-09-19T06:36:31.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:30 smithi155 ceph-mon[35292]: pgmap v810: 129 pgs: 129 active+clean; 444 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 5.0 KiB/s rd, 4.2 MiB/s wr, 125 op/s 2024-09-19T06:36:31.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:30 smithi053 ceph-mon[27968]: pgmap v810: 129 pgs: 129 active+clean; 444 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 5.0 KiB/s rd, 4.2 MiB/s wr, 125 op/s 2024-09-19T06:36:31.855 INFO:tasks.workunit.client.0.smithi053.stdout: 10 195910 27.75 MB/sec execute 255 sec latency 95.236 ms 2024-09-19T06:36:32.036 INFO:tasks.workunit.client.1.smithi155.stdout: 10 50500 7.16 MB/sec execute 258 sec latency 101.711 ms 2024-09-19T06:36:32.855 INFO:tasks.workunit.client.0.smithi053.stdout: 10 196836 27.82 MB/sec execute 256 sec latency 102.405 ms 2024-09-19T06:36:33.037 INFO:tasks.workunit.client.1.smithi155.stdout: 10 50605 7.15 MB/sec execute 259 sec latency 220.121 ms 2024-09-19T06:36:33.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:32 smithi053 ceph-mon[27968]: pgmap v811: 129 pgs: 129 active+clean; 445 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.1 MiB/s wr, 105 op/s 2024-09-19T06:36:33.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:32 smithi155 ceph-mon[35292]: pgmap v811: 129 pgs: 129 active+clean; 445 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.1 MiB/s wr, 105 op/s 2024-09-19T06:36:33.857 INFO:tasks.workunit.client.0.smithi053.stdout: 10 198118 28.01 MB/sec execute 257 sec latency 122.870 ms 2024-09-19T06:36:34.036 INFO:tasks.workunit.client.1.smithi155.stdout: 10 50761 7.14 MB/sec execute 260 sec latency 92.057 ms 2024-09-19T06:36:34.855 INFO:tasks.workunit.client.0.smithi053.stdout: 10 199334 28.15 MB/sec execute 258 sec latency 124.954 ms 2024-09-19T06:36:35.036 INFO:tasks.workunit.client.1.smithi155.stdout: 10 50919 7.15 MB/sec execute 261 sec latency 108.340 ms 2024-09-19T06:36:35.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:34 smithi155 ceph-mon[35292]: pgmap v812: 129 pgs: 129 active+clean; 438 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 9.0 KiB/s rd, 4.4 MiB/s wr, 192 op/s 2024-09-19T06:36:35.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:34 smithi053 ceph-mon[27968]: pgmap v812: 129 pgs: 129 active+clean; 438 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 9.0 KiB/s rd, 4.4 MiB/s wr, 192 op/s 2024-09-19T06:36:35.855 INFO:tasks.workunit.client.0.smithi053.stdout: 10 199454 28.06 MB/sec execute 259 sec latency 209.233 ms 2024-09-19T06:36:36.036 INFO:tasks.workunit.client.1.smithi155.stdout: 10 51081 7.17 MB/sec execute 262 sec latency 100.552 ms 2024-09-19T06:36:36.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:35 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:36:36.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:35 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:36.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:35 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:36:36.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:35 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:36:36.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:35 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:36:36.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:35 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:36.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:35 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:36:36.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:35 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:36:36.856 INFO:tasks.workunit.client.0.smithi053.stdout: 10 200237 28.13 MB/sec execute 260 sec latency 250.303 ms 2024-09-19T06:36:37.037 INFO:tasks.workunit.client.1.smithi155.stdout: 10 51273 7.19 MB/sec execute 263 sec latency 92.814 ms 2024-09-19T06:36:37.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:36 smithi155 ceph-mon[35292]: pgmap v813: 129 pgs: 129 active+clean; 436 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 7.2 KiB/s rd, 3.6 MiB/s wr, 233 op/s 2024-09-19T06:36:37.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:36 smithi053 ceph-mon[27968]: pgmap v813: 129 pgs: 129 active+clean; 436 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 7.2 KiB/s rd, 3.6 MiB/s wr, 233 op/s 2024-09-19T06:36:37.855 INFO:tasks.workunit.client.0.smithi053.stdout: 10 201047 28.11 MB/sec execute 261 sec latency 308.334 ms 2024-09-19T06:36:38.037 INFO:tasks.workunit.client.1.smithi155.stdout: 10 51431 7.17 MB/sec execute 264 sec latency 103.291 ms 2024-09-19T06:36:38.855 INFO:tasks.workunit.client.0.smithi053.stdout: 10 202130 28.30 MB/sec execute 262 sec latency 130.269 ms 2024-09-19T06:36:39.034 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:38 smithi155 ceph-mon[35292]: pgmap v814: 129 pgs: 129 active+clean; 432 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 7.2 KiB/s rd, 3.6 MiB/s wr, 274 op/s 2024-09-19T06:36:39.038 INFO:tasks.workunit.client.1.smithi155.stdout: 10 51610 7.15 MB/sec execute 265 sec latency 80.781 ms 2024-09-19T06:36:39.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:38 smithi053 ceph-mon[27968]: pgmap v814: 129 pgs: 129 active+clean; 432 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 7.2 KiB/s rd, 3.6 MiB/s wr, 274 op/s 2024-09-19T06:36:39.856 INFO:tasks.workunit.client.0.smithi053.stdout: 10 202940 28.35 MB/sec execute 263 sec latency 96.571 ms 2024-09-19T06:36:40.038 INFO:tasks.workunit.client.1.smithi155.stdout: 10 51784 7.14 MB/sec execute 266 sec latency 99.562 ms 2024-09-19T06:36:40.855 INFO:tasks.workunit.client.0.smithi053.stdout: 10 203173 28.28 MB/sec execute 264 sec latency 318.885 ms 2024-09-19T06:36:41.038 INFO:tasks.workunit.client.1.smithi155.stdout: 10 51973 7.15 MB/sec execute 267 sec latency 95.333 ms 2024-09-19T06:36:41.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:40 smithi053 ceph-mon[27968]: pgmap v815: 129 pgs: 129 active+clean; 428 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 9.4 KiB/s rd, 3.6 MiB/s wr, 322 op/s 2024-09-19T06:36:41.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:40 smithi155 ceph-mon[35292]: pgmap v815: 129 pgs: 129 active+clean; 428 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 9.4 KiB/s rd, 3.6 MiB/s wr, 322 op/s 2024-09-19T06:36:41.856 INFO:tasks.workunit.client.0.smithi053.stdout: 10 204287 28.35 MB/sec execute 265 sec latency 179.610 ms 2024-09-19T06:36:42.039 INFO:tasks.workunit.client.1.smithi155.stdout: 10 52150 7.15 MB/sec execute 268 sec latency 136.888 ms 2024-09-19T06:36:42.856 INFO:tasks.workunit.client.0.smithi053.stdout: 10 205062 28.46 MB/sec execute 266 sec latency 373.509 ms 2024-09-19T06:36:43.040 INFO:tasks.workunit.client.1.smithi155.stdout: 10 52296 7.16 MB/sec execute 269 sec latency 90.141 ms 2024-09-19T06:36:43.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:42 smithi053 ceph-mon[27968]: pgmap v816: 129 pgs: 129 active+clean; 428 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 8.0 KiB/s rd, 2.9 MiB/s wr, 300 op/s 2024-09-19T06:36:43.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:42 smithi155 ceph-mon[35292]: pgmap v816: 129 pgs: 129 active+clean; 428 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 8.0 KiB/s rd, 2.9 MiB/s wr, 300 op/s 2024-09-19T06:36:43.856 INFO:tasks.workunit.client.0.smithi053.stdout: 10 205273 28.40 MB/sec execute 267 sec latency 526.595 ms 2024-09-19T06:36:44.039 INFO:tasks.workunit.client.1.smithi155.stdout: 10 52398 7.16 MB/sec execute 270 sec latency 190.439 ms 2024-09-19T06:36:44.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:43 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:36:44.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:43 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:36:44.856 INFO:tasks.workunit.client.0.smithi053.stdout: 10 205579 28.36 MB/sec execute 268 sec latency 428.688 ms 2024-09-19T06:36:45.040 INFO:tasks.workunit.client.1.smithi155.stdout: 10 52547 7.19 MB/sec execute 271 sec latency 145.658 ms 2024-09-19T06:36:45.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:44 smithi155 ceph-mon[35292]: pgmap v817: 129 pgs: 129 active+clean; 436 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 9.6 KiB/s rd, 3.6 MiB/s wr, 349 op/s 2024-09-19T06:36:45.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:44 smithi053 ceph-mon[27968]: pgmap v817: 129 pgs: 129 active+clean; 436 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 9.6 KiB/s rd, 3.6 MiB/s wr, 349 op/s 2024-09-19T06:36:45.856 INFO:tasks.workunit.client.0.smithi053.stdout: 10 205694 28.28 MB/sec execute 269 sec latency 467.593 ms 2024-09-19T06:36:46.040 INFO:tasks.workunit.client.1.smithi155.stdout: 10 52691 7.22 MB/sec execute 272 sec latency 198.233 ms 2024-09-19T06:36:46.856 INFO:tasks.workunit.client.0.smithi053.stdout: 10 206408 28.32 MB/sec execute 270 sec latency 534.362 ms 2024-09-19T06:36:47.040 INFO:tasks.workunit.client.1.smithi155.stdout: 10 52862 7.24 MB/sec execute 273 sec latency 255.349 ms 2024-09-19T06:36:47.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:46 smithi155 ceph-mon[35292]: pgmap v818: 129 pgs: 129 active+clean; 437 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 3.8 KiB/s rd, 3.2 MiB/s wr, 265 op/s 2024-09-19T06:36:47.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:46 smithi053 ceph-mon[27968]: pgmap v818: 129 pgs: 129 active+clean; 437 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 3.8 KiB/s rd, 3.2 MiB/s wr, 265 op/s 2024-09-19T06:36:47.856 INFO:tasks.workunit.client.0.smithi053.stdout: 10 206423 28.21 MB/sec execute 271 sec latency 553.022 ms 2024-09-19T06:36:48.040 INFO:tasks.workunit.client.1.smithi155.stdout: 10 52992 7.24 MB/sec execute 274 sec latency 155.253 ms 2024-09-19T06:36:48.856 INFO:tasks.workunit.client.0.smithi053.stdout: 10 206439 28.11 MB/sec execute 272 sec latency 392.333 ms 2024-09-19T06:36:49.040 INFO:tasks.workunit.client.1.smithi155.stdout: 10 53061 7.24 MB/sec execute 275 sec latency 145.824 ms 2024-09-19T06:36:49.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:48 smithi155 ceph-mon[35292]: pgmap v819: 129 pgs: 129 active+clean; 442 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 3.8 KiB/s rd, 3.2 MiB/s wr, 210 op/s 2024-09-19T06:36:49.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:48 smithi053 ceph-mon[27968]: pgmap v819: 129 pgs: 129 active+clean; 442 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 3.8 KiB/s rd, 3.2 MiB/s wr, 210 op/s 2024-09-19T06:36:49.856 INFO:tasks.workunit.client.0.smithi053.stdout: 10 206475 28.01 MB/sec execute 273 sec latency 392.653 ms 2024-09-19T06:36:50.040 INFO:tasks.workunit.client.1.smithi155.stdout: 10 53096 7.22 MB/sec execute 276 sec latency 182.577 ms 2024-09-19T06:36:50.856 INFO:tasks.workunit.client.0.smithi053.stdout: 10 206496 27.91 MB/sec execute 274 sec latency 450.534 ms 2024-09-19T06:36:51.040 INFO:tasks.workunit.client.1.smithi155.stdout: 10 53138 7.20 MB/sec execute 277 sec latency 159.486 ms 2024-09-19T06:36:51.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:50 smithi155 ceph-mon[35292]: pgmap v820: 129 pgs: 129 active+clean; 450 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 5.7 KiB/s rd, 3.7 MiB/s wr, 176 op/s 2024-09-19T06:36:51.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:50 smithi053 ceph-mon[27968]: pgmap v820: 129 pgs: 129 active+clean; 450 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 5.7 KiB/s rd, 3.7 MiB/s wr, 176 op/s 2024-09-19T06:36:51.856 INFO:tasks.workunit.client.0.smithi053.stdout: 10 206542 27.81 MB/sec execute 275 sec latency 392.256 ms 2024-09-19T06:36:52.040 INFO:tasks.workunit.client.1.smithi155.stdout: 10 53244 7.21 MB/sec execute 278 sec latency 196.966 ms 2024-09-19T06:36:52.856 INFO:tasks.workunit.client.0.smithi053.stdout: 10 206558 27.71 MB/sec execute 276 sec latency 443.876 ms 2024-09-19T06:36:53.040 INFO:tasks.workunit.client.1.smithi155.stdout: 10 53358 7.22 MB/sec execute 279 sec latency 158.854 ms 2024-09-19T06:36:53.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:52 smithi155 ceph-mon[35292]: pgmap v821: 129 pgs: 129 active+clean; 453 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 3.4 KiB/s rd, 3.4 MiB/s wr, 114 op/s 2024-09-19T06:36:53.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:52 smithi053 ceph-mon[27968]: pgmap v821: 129 pgs: 129 active+clean; 453 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 3.4 KiB/s rd, 3.4 MiB/s wr, 114 op/s 2024-09-19T06:36:53.856 INFO:tasks.workunit.client.0.smithi053.stdout: 10 206644 27.62 MB/sec execute 277 sec latency 408.970 ms 2024-09-19T06:36:54.040 INFO:tasks.workunit.client.1.smithi155.stdout: 10 53489 7.23 MB/sec execute 280 sec latency 179.848 ms 2024-09-19T06:36:54.856 INFO:tasks.workunit.client.0.smithi053.stdout: 10 206811 27.56 MB/sec execute 278 sec latency 366.084 ms 2024-09-19T06:36:55.040 INFO:tasks.workunit.client.1.smithi155.stdout: 10 53584 7.22 MB/sec execute 281 sec latency 166.398 ms 2024-09-19T06:36:55.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:54 smithi155 ceph-mon[35292]: pgmap v822: 129 pgs: 129 active+clean; 457 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 4.8 KiB/s rd, 4.0 MiB/s wr, 135 op/s 2024-09-19T06:36:55.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:54 smithi053 ceph-mon[27968]: pgmap v822: 129 pgs: 129 active+clean; 457 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 4.8 KiB/s rd, 4.0 MiB/s wr, 135 op/s 2024-09-19T06:36:55.856 INFO:tasks.workunit.client.0.smithi053.stdout: 10 207036 27.49 MB/sec execute 279 sec latency 409.074 ms 2024-09-19T06:36:56.040 INFO:tasks.workunit.client.1.smithi155.stdout: 10 53663 7.20 MB/sec execute 282 sec latency 202.343 ms 2024-09-19T06:36:56.856 INFO:tasks.workunit.client.0.smithi053.stdout: 10 207515 27.49 MB/sec execute 280 sec latency 484.115 ms 2024-09-19T06:36:57.040 INFO:tasks.workunit.client.1.smithi155.stdout: 10 53737 7.18 MB/sec execute 283 sec latency 177.222 ms 2024-09-19T06:36:57.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:56 smithi155 ceph-mon[35292]: pgmap v823: 129 pgs: 129 active+clean; 458 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 3.3 MiB/s wr, 84 op/s 2024-09-19T06:36:57.255 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:56 smithi053 ceph-mon[27968]: pgmap v823: 129 pgs: 129 active+clean; 458 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 3.3 MiB/s wr, 84 op/s 2024-09-19T06:36:57.856 INFO:tasks.workunit.client.0.smithi053.stdout: 10 207722 27.41 MB/sec execute 281 sec latency 449.487 ms 2024-09-19T06:36:58.040 INFO:tasks.workunit.client.1.smithi155.stdout: 10 53818 7.16 MB/sec execute 284 sec latency 201.942 ms 2024-09-19T06:36:58.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:57 smithi155 ceph-mon[35292]: pgmap v824: 129 pgs: 129 active+clean; 459 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 3.4 MiB/s wr, 83 op/s 2024-09-19T06:36:58.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:57 smithi053 ceph-mon[27968]: pgmap v824: 129 pgs: 129 active+clean; 459 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 3.4 MiB/s wr, 83 op/s 2024-09-19T06:36:58.856 INFO:tasks.workunit.client.0.smithi053.stdout: 10 208144 27.37 MB/sec execute 282 sec latency 468.118 ms 2024-09-19T06:36:59.041 INFO:tasks.workunit.client.1.smithi155.stdout: 10 53895 7.14 MB/sec execute 285 sec latency 194.721 ms 2024-09-19T06:36:59.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:58 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:36:59.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:58 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:36:59.856 INFO:tasks.workunit.client.0.smithi053.stdout: 10 208359 27.34 MB/sec execute 283 sec latency 464.366 ms 2024-09-19T06:37:00.040 INFO:tasks.workunit.client.1.smithi155.stdout: 10 53996 7.12 MB/sec execute 286 sec latency 199.892 ms 2024-09-19T06:37:00.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:36:59 smithi155 ceph-mon[35292]: pgmap v825: 129 pgs: 129 active+clean; 461 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 6.2 KiB/s rd, 3.5 MiB/s wr, 93 op/s 2024-09-19T06:37:00.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:36:59 smithi053 ceph-mon[27968]: pgmap v825: 129 pgs: 129 active+clean; 461 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 6.2 KiB/s rd, 3.5 MiB/s wr, 93 op/s 2024-09-19T06:37:00.856 INFO:tasks.workunit.client.0.smithi053.stdout: 10 209121 27.44 MB/sec execute 284 sec latency 324.518 ms 2024-09-19T06:37:01.040 INFO:tasks.workunit.client.1.smithi155.stdout: 10 54143 7.11 MB/sec execute 287 sec latency 153.955 ms 2024-09-19T06:37:01.856 INFO:tasks.workunit.client.0.smithi053.stdout: 10 210030 27.50 MB/sec execute 285 sec latency 108.307 ms 2024-09-19T06:37:02.040 INFO:tasks.workunit.client.1.smithi155.stdout: 10 54307 7.12 MB/sec execute 288 sec latency 124.303 ms 2024-09-19T06:37:02.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:02 smithi155 ceph-mon[35292]: pgmap v826: 129 pgs: 129 active+clean; 461 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 4.4 KiB/s rd, 2.8 MiB/s wr, 83 op/s 2024-09-19T06:37:02.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:02 smithi053 ceph-mon[27968]: pgmap v826: 129 pgs: 129 active+clean; 461 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 4.4 KiB/s rd, 2.8 MiB/s wr, 83 op/s 2024-09-19T06:37:02.857 INFO:tasks.workunit.client.0.smithi053.stdout: 10 210265 27.44 MB/sec execute 286 sec latency 166.956 ms 2024-09-19T06:37:03.040 INFO:tasks.workunit.client.1.smithi155.stdout: 10 54460 7.12 MB/sec execute 289 sec latency 94.617 ms 2024-09-19T06:37:03.857 INFO:tasks.workunit.client.0.smithi053.stdout: 10 211709 27.56 MB/sec execute 287 sec latency 142.295 ms 2024-09-19T06:37:04.040 INFO:tasks.workunit.client.1.smithi155.stdout: 10 54646 7.14 MB/sec execute 290 sec latency 92.421 ms 2024-09-19T06:37:04.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:04 smithi155 ceph-mon[35292]: pgmap v827: 129 pgs: 129 active+clean; 451 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 8.7 KiB/s rd, 3.1 MiB/s wr, 141 op/s 2024-09-19T06:37:04.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:04 smithi053 ceph-mon[27968]: pgmap v827: 129 pgs: 129 active+clean; 451 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 8.7 KiB/s rd, 3.1 MiB/s wr, 141 op/s 2024-09-19T06:37:04.857 INFO:tasks.workunit.client.0.smithi053.stdout: 10 212781 27.74 MB/sec execute 288 sec latency 118.853 ms 2024-09-19T06:37:05.040 INFO:tasks.workunit.client.1.smithi155.stdout: 10 54822 7.16 MB/sec execute 291 sec latency 90.394 ms 2024-09-19T06:37:05.857 INFO:tasks.workunit.client.0.smithi053.stdout: 10 213564 27.78 MB/sec execute 289 sec latency 158.556 ms 2024-09-19T06:37:06.041 INFO:tasks.workunit.client.1.smithi155.stdout: 10 54967 7.14 MB/sec execute 292 sec latency 152.782 ms 2024-09-19T06:37:06.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:06 smithi155 ceph-mon[35292]: pgmap v828: 129 pgs: 129 active+clean; 445 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.3 KiB/s rd, 2.7 MiB/s wr, 132 op/s 2024-09-19T06:37:06.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:06 smithi053 ceph-mon[27968]: pgmap v828: 129 pgs: 129 active+clean; 445 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.3 KiB/s rd, 2.7 MiB/s wr, 132 op/s 2024-09-19T06:37:06.857 INFO:tasks.workunit.client.0.smithi053.stdout: 10 213586 27.69 MB/sec execute 290 sec latency 370.689 ms 2024-09-19T06:37:07.041 INFO:tasks.workunit.client.1.smithi155.stdout: 10 55092 7.12 MB/sec execute 293 sec latency 191.461 ms 2024-09-19T06:37:07.857 INFO:tasks.workunit.client.0.smithi053.stdout: 10 213625 27.60 MB/sec execute 291 sec latency 400.910 ms 2024-09-19T06:37:08.041 INFO:tasks.workunit.client.1.smithi155.stdout: 10 55201 7.10 MB/sec execute 294 sec latency 216.039 ms 2024-09-19T06:37:08.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:08 smithi155 ceph-mon[35292]: pgmap v829: 129 pgs: 129 active+clean; 425 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 7.3 KiB/s rd, 2.9 MiB/s wr, 175 op/s 2024-09-19T06:37:08.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:08 smithi053 ceph-mon[27968]: pgmap v829: 129 pgs: 129 active+clean; 425 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 7.3 KiB/s rd, 2.9 MiB/s wr, 175 op/s 2024-09-19T06:37:08.857 INFO:tasks.workunit.client.0.smithi053.stdout: 10 213647 27.51 MB/sec execute 292 sec latency 484.071 ms 2024-09-19T06:37:09.041 INFO:tasks.workunit.client.1.smithi155.stdout: 10 55318 7.09 MB/sec execute 295 sec latency 187.468 ms 2024-09-19T06:37:09.857 INFO:tasks.workunit.client.0.smithi053.stdout: 10 213717 27.42 MB/sec execute 293 sec latency 536.210 ms 2024-09-19T06:37:10.041 INFO:tasks.workunit.client.1.smithi155.stdout: 10 55460 7.08 MB/sec execute 296 sec latency 139.597 ms 2024-09-19T06:37:10.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:10 smithi155 ceph-mon[35292]: pgmap v830: 129 pgs: 129 active+clean; 420 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 9.4 KiB/s rd, 2.7 MiB/s wr, 190 op/s 2024-09-19T06:37:10.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:10 smithi053 ceph-mon[27968]: pgmap v830: 129 pgs: 129 active+clean; 420 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 9.4 KiB/s rd, 2.7 MiB/s wr, 190 op/s 2024-09-19T06:37:10.857 INFO:tasks.workunit.client.0.smithi053.stdout: 10 213820 27.34 MB/sec execute 294 sec latency 467.637 ms 2024-09-19T06:37:11.041 INFO:tasks.workunit.client.1.smithi155.stdout: 10 55583 7.08 MB/sec execute 297 sec latency 181.498 ms 2024-09-19T06:37:11.857 INFO:tasks.workunit.client.0.smithi053.stdout: 10 214040 27.30 MB/sec execute 295 sec latency 492.836 ms 2024-09-19T06:37:12.041 INFO:tasks.workunit.client.1.smithi155.stdout: 10 55719 7.09 MB/sec execute 298 sec latency 160.182 ms 2024-09-19T06:37:12.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:12 smithi155 ceph-mon[35292]: pgmap v831: 129 pgs: 129 active+clean; 420 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 6.4 KiB/s rd, 2.1 MiB/s wr, 162 op/s 2024-09-19T06:37:12.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:12 smithi053 ceph-mon[27968]: pgmap v831: 129 pgs: 129 active+clean; 420 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 6.4 KiB/s rd, 2.1 MiB/s wr, 162 op/s 2024-09-19T06:37:12.857 INFO:tasks.workunit.client.0.smithi053.stdout: 10 214511 27.29 MB/sec execute 296 sec latency 596.966 ms 2024-09-19T06:37:13.041 INFO:tasks.workunit.client.1.smithi155.stdout: 10 55803 7.08 MB/sec execute 299 sec latency 170.907 ms 2024-09-19T06:37:13.857 INFO:tasks.workunit.client.0.smithi053.stdout: 10 214822 27.22 MB/sec execute 297 sec latency 460.052 ms 2024-09-19T06:37:14.041 INFO:tasks.workunit.client.1.smithi155.stdout: 10 55914 7.07 MB/sec execute 300 sec latency 198.729 ms 2024-09-19T06:37:14.152 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:14 smithi155 ceph-mon[35292]: pgmap v832: 129 pgs: 129 active+clean; 423 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 7.7 KiB/s rd, 2.6 MiB/s wr, 199 op/s 2024-09-19T06:37:14.152 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:14 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:37:14.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:14 smithi053 ceph-mon[27968]: pgmap v832: 129 pgs: 129 active+clean; 423 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 7.7 KiB/s rd, 2.6 MiB/s wr, 199 op/s 2024-09-19T06:37:14.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:14 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:37:14.857 INFO:tasks.workunit.client.0.smithi053.stdout: 10 215191 27.17 MB/sec execute 298 sec latency 494.327 ms 2024-09-19T06:37:15.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 56069 7.10 MB/sec execute 301 sec latency 150.050 ms 2024-09-19T06:37:15.857 INFO:tasks.workunit.client.0.smithi053.stdout: 10 215375 27.11 MB/sec execute 299 sec latency 482.867 ms 2024-09-19T06:37:16.043 INFO:tasks.workunit.client.1.smithi155.stdout: 10 56205 7.13 MB/sec execute 302 sec latency 177.982 ms 2024-09-19T06:37:16.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:16 smithi155 ceph-mon[35292]: pgmap v833: 129 pgs: 129 active+clean; 427 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.6 MiB/s wr, 150 op/s 2024-09-19T06:37:16.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:16 smithi053 ceph-mon[27968]: pgmap v833: 129 pgs: 129 active+clean; 427 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.6 MiB/s wr, 150 op/s 2024-09-19T06:37:16.857 INFO:tasks.workunit.client.0.smithi053.stdout: 10 215812 27.17 MB/sec execute 300 sec latency 539.802 ms 2024-09-19T06:37:17.041 INFO:tasks.workunit.client.1.smithi155.stdout: 10 56420 7.16 MB/sec execute 303 sec latency 129.685 ms 2024-09-19T06:37:17.857 INFO:tasks.workunit.client.0.smithi053.stdout: 10 217111 27.31 MB/sec execute 301 sec latency 295.946 ms 2024-09-19T06:37:18.041 INFO:tasks.workunit.client.1.smithi155.stdout: 10 56586 7.17 MB/sec execute 304 sec latency 117.509 ms 2024-09-19T06:37:18.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:17 smithi053 ceph-mon[27968]: pgmap v834: 129 pgs: 129 active+clean; 433 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.8 MiB/s wr, 145 op/s 2024-09-19T06:37:18.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:17 smithi155 ceph-mon[35292]: pgmap v834: 129 pgs: 129 active+clean; 433 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.8 MiB/s wr, 145 op/s 2024-09-19T06:37:18.857 INFO:tasks.workunit.client.0.smithi053.stdout: 10 217303 27.23 MB/sec execute 302 sec latency 83.683 ms 2024-09-19T06:37:19.041 INFO:tasks.workunit.client.1.smithi155.stdout: 10 56645 7.16 MB/sec execute 305 sec latency 171.691 ms 2024-09-19T06:37:19.857 INFO:tasks.workunit.client.0.smithi053.stdout: 10 218413 27.31 MB/sec execute 303 sec latency 116.168 ms 2024-09-19T06:37:20.041 INFO:tasks.workunit.client.1.smithi155.stdout: 10 56711 7.15 MB/sec execute 306 sec latency 89.857 ms 2024-09-19T06:37:20.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:20 smithi155 ceph-mon[35292]: pgmap v835: 129 pgs: 129 active+clean; 442 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 4.9 KiB/s rd, 3.2 MiB/s wr, 111 op/s 2024-09-19T06:37:20.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:20 smithi053 ceph-mon[27968]: pgmap v835: 129 pgs: 129 active+clean; 442 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 4.9 KiB/s rd, 3.2 MiB/s wr, 111 op/s 2024-09-19T06:37:20.857 INFO:tasks.workunit.client.0.smithi053.stdout: 10 219470 27.45 MB/sec execute 304 sec latency 275.578 ms 2024-09-19T06:37:21.041 INFO:tasks.workunit.client.1.smithi155.stdout: 10 56861 7.17 MB/sec execute 307 sec latency 97.537 ms 2024-09-19T06:37:21.857 INFO:tasks.workunit.client.0.smithi053.stdout: 10 220694 27.58 MB/sec execute 305 sec latency 125.111 ms 2024-09-19T06:37:22.041 INFO:tasks.workunit.client.1.smithi155.stdout: 10 57036 7.19 MB/sec execute 308 sec latency 116.252 ms 2024-09-19T06:37:22.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:22 smithi155 ceph-mon[35292]: pgmap v836: 129 pgs: 129 active+clean; 446 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 2.8 KiB/s rd, 3.3 MiB/s wr, 105 op/s 2024-09-19T06:37:22.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:22 smithi053 ceph-mon[27968]: pgmap v836: 129 pgs: 129 active+clean; 446 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 2.8 KiB/s rd, 3.3 MiB/s wr, 105 op/s 2024-09-19T06:37:22.857 INFO:tasks.workunit.client.0.smithi053.stdout: 10 220721 27.49 MB/sec execute 306 sec latency 370.034 ms 2024-09-19T06:37:23.041 INFO:tasks.workunit.client.1.smithi155.stdout: 10 57121 7.18 MB/sec execute 309 sec latency 167.687 ms 2024-09-19T06:37:23.857 INFO:tasks.workunit.client.0.smithi053.stdout: 10 220769 27.41 MB/sec execute 307 sec latency 438.779 ms 2024-09-19T06:37:24.041 INFO:tasks.workunit.client.1.smithi155.stdout: 10 57199 7.17 MB/sec execute 310 sec latency 248.456 ms 2024-09-19T06:37:24.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:24 smithi155 ceph-mon[35292]: pgmap v837: 129 pgs: 129 active+clean; 449 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 4.7 KiB/s rd, 4.2 MiB/s wr, 121 op/s 2024-09-19T06:37:24.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:24 smithi053 ceph-mon[27968]: pgmap v837: 129 pgs: 129 active+clean; 449 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 4.7 KiB/s rd, 4.2 MiB/s wr, 121 op/s 2024-09-19T06:37:24.857 INFO:tasks.workunit.client.0.smithi053.stdout: 10 220782 27.32 MB/sec execute 308 sec latency 450.502 ms 2024-09-19T06:37:25.041 INFO:tasks.workunit.client.1.smithi155.stdout: 10 57267 7.15 MB/sec execute 311 sec latency 306.345 ms 2024-09-19T06:37:25.857 INFO:tasks.workunit.client.0.smithi053.stdout: 10 220856 27.24 MB/sec execute 309 sec latency 408.975 ms 2024-09-19T06:37:26.041 INFO:tasks.workunit.client.1.smithi155.stdout: 10 57334 7.13 MB/sec execute 312 sec latency 295.004 ms 2024-09-19T06:37:26.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:26 smithi155 ceph-mon[35292]: pgmap v838: 129 pgs: 129 active+clean; 450 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 3.5 KiB/s rd, 4.2 MiB/s wr, 103 op/s 2024-09-19T06:37:26.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:26 smithi053 ceph-mon[27968]: pgmap v838: 129 pgs: 129 active+clean; 450 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 3.5 KiB/s rd, 4.2 MiB/s wr, 103 op/s 2024-09-19T06:37:26.858 INFO:tasks.workunit.client.0.smithi053.stdout: 10 220905 27.15 MB/sec execute 310 sec latency 442.744 ms 2024-09-19T06:37:27.041 INFO:tasks.workunit.client.1.smithi155.stdout: 10 57414 7.11 MB/sec execute 313 sec latency 174.480 ms 2024-09-19T06:37:27.857 INFO:tasks.workunit.client.0.smithi053.stdout: 10 221903 27.22 MB/sec execute 311 sec latency 441.267 ms 2024-09-19T06:37:28.041 INFO:tasks.workunit.client.1.smithi155.stdout: 10 57526 7.10 MB/sec execute 314 sec latency 171.947 ms 2024-09-19T06:37:28.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:27 smithi053 ceph-mon[27968]: pgmap v839: 129 pgs: 129 active+clean; 451 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 3.5 KiB/s rd, 3.9 MiB/s wr, 97 op/s 2024-09-19T06:37:28.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:27 smithi155 ceph-mon[35292]: pgmap v839: 129 pgs: 129 active+clean; 451 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 3.5 KiB/s rd, 3.9 MiB/s wr, 97 op/s 2024-09-19T06:37:28.858 INFO:tasks.workunit.client.0.smithi053.stdout: 10 223009 27.36 MB/sec execute 312 sec latency 141.271 ms 2024-09-19T06:37:29.041 INFO:tasks.workunit.client.1.smithi155.stdout: 10 57667 7.09 MB/sec execute 315 sec latency 126.273 ms 2024-09-19T06:37:29.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:28 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:37:29.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:28 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:37:29.858 INFO:tasks.workunit.client.0.smithi053.stdout: 10 224242 27.49 MB/sec execute 313 sec latency 200.863 ms 2024-09-19T06:37:30.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 57810 7.08 MB/sec execute 316 sec latency 139.622 ms 2024-09-19T06:37:30.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:30 smithi053 ceph-mon[27968]: pgmap v840: 129 pgs: 129 active+clean; 451 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 6.9 KiB/s rd, 3.7 MiB/s wr, 110 op/s 2024-09-19T06:37:30.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:30 smithi155 ceph-mon[35292]: pgmap v840: 129 pgs: 129 active+clean; 451 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 6.9 KiB/s rd, 3.7 MiB/s wr, 110 op/s 2024-09-19T06:37:30.858 INFO:tasks.workunit.client.0.smithi053.stdout: 10 224407 27.41 MB/sec execute 314 sec latency 209.166 ms 2024-09-19T06:37:31.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 57963 7.09 MB/sec execute 317 sec latency 109.967 ms 2024-09-19T06:37:31.858 INFO:tasks.workunit.client.0.smithi053.stdout: 10 225378 27.47 MB/sec execute 315 sec latency 413.447 ms 2024-09-19T06:37:32.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 58117 7.07 MB/sec execute 318 sec latency 92.276 ms 2024-09-19T06:37:32.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:32 smithi155 ceph-mon[35292]: pgmap v841: 129 pgs: 129 active+clean; 452 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 5.3 KiB/s rd, 2.9 MiB/s wr, 103 op/s 2024-09-19T06:37:32.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:32 smithi053 ceph-mon[27968]: pgmap v841: 129 pgs: 129 active+clean; 452 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 5.3 KiB/s rd, 2.9 MiB/s wr, 103 op/s 2024-09-19T06:37:32.858 INFO:tasks.workunit.client.0.smithi053.stdout: 10 226058 27.48 MB/sec execute 316 sec latency 374.771 ms 2024-09-19T06:37:33.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 58292 7.11 MB/sec execute 319 sec latency 169.923 ms 2024-09-19T06:37:33.858 INFO:tasks.workunit.client.0.smithi053.stdout: 10 226455 27.52 MB/sec execute 317 sec latency 548.951 ms 2024-09-19T06:37:34.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 58409 7.11 MB/sec execute 320 sec latency 208.267 ms 2024-09-19T06:37:34.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:34 smithi155 ceph-mon[35292]: pgmap v842: 129 pgs: 129 active+clean; 439 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 9.2 KiB/s rd, 3.5 MiB/s wr, 160 op/s 2024-09-19T06:37:34.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:34 smithi053 ceph-mon[27968]: pgmap v842: 129 pgs: 129 active+clean; 439 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 9.2 KiB/s rd, 3.5 MiB/s wr, 160 op/s 2024-09-19T06:37:34.858 INFO:tasks.workunit.client.0.smithi053.stdout: 10 226706 27.47 MB/sec execute 318 sec latency 516.090 ms 2024-09-19T06:37:35.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 58524 7.09 MB/sec execute 321 sec latency 153.325 ms 2024-09-19T06:37:35.858 INFO:tasks.workunit.client.0.smithi053.stdout: 10 226823 27.41 MB/sec execute 319 sec latency 428.569 ms 2024-09-19T06:37:36.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 58652 7.08 MB/sec execute 322 sec latency 161.911 ms 2024-09-19T06:37:36.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:36 smithi155 ceph-mon[35292]: pgmap v843: 129 pgs: 129 active+clean; 433 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.3 KiB/s rd, 2.8 MiB/s wr, 158 op/s 2024-09-19T06:37:36.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:36 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:37:36.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:36 smithi053 ceph-mon[27968]: pgmap v843: 129 pgs: 129 active+clean; 433 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.3 KiB/s rd, 2.8 MiB/s wr, 158 op/s 2024-09-19T06:37:36.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:36 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:37:36.858 INFO:tasks.workunit.client.0.smithi053.stdout: 10 227643 27.46 MB/sec execute 320 sec latency 401.450 ms 2024-09-19T06:37:37.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 58791 7.06 MB/sec execute 323 sec latency 126.897 ms 2024-09-19T06:37:37.858 INFO:tasks.workunit.client.0.smithi053.stdout: 10 227783 27.40 MB/sec execute 321 sec latency 382.808 ms 2024-09-19T06:37:38.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 58940 7.05 MB/sec execute 324 sec latency 166.637 ms 2024-09-19T06:37:38.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:38 smithi053 ceph-mon[27968]: pgmap v844: 129 pgs: 129 active+clean; 436 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.3 KiB/s rd, 2.4 MiB/s wr, 171 op/s 2024-09-19T06:37:38.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:38 smithi155 ceph-mon[35292]: pgmap v844: 129 pgs: 129 active+clean; 436 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.3 KiB/s rd, 2.4 MiB/s wr, 171 op/s 2024-09-19T06:37:38.858 INFO:tasks.workunit.client.0.smithi053.stdout: 10 227865 27.32 MB/sec execute 322 sec latency 134.862 ms 2024-09-19T06:37:39.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 59114 7.05 MB/sec execute 325 sec latency 98.763 ms 2024-09-19T06:37:39.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:39 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:37:39.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:39 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:37:39.858 INFO:tasks.workunit.client.0.smithi053.stdout: 10 228705 27.38 MB/sec execute 323 sec latency 117.436 ms 2024-09-19T06:37:39.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:39 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:37:39.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:39 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:37:40.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 59286 7.07 MB/sec execute 326 sec latency 102.285 ms 2024-09-19T06:37:40.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:40 smithi053 ceph-mon[27968]: pgmap v845: 129 pgs: 129 active+clean; 437 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 8.9 KiB/s rd, 2.5 MiB/s wr, 187 op/s 2024-09-19T06:37:40.858 INFO:tasks.workunit.client.0.smithi053.stdout: 10 229817 27.47 MB/sec execute 324 sec latency 212.287 ms 2024-09-19T06:37:40.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:40 smithi155 ceph-mon[35292]: pgmap v845: 129 pgs: 129 active+clean; 437 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 8.9 KiB/s rd, 2.5 MiB/s wr, 187 op/s 2024-09-19T06:37:41.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 59413 7.07 MB/sec execute 327 sec latency 137.364 ms 2024-09-19T06:37:41.858 INFO:tasks.workunit.client.0.smithi053.stdout: 10 231193 27.63 MB/sec execute 325 sec latency 141.194 ms 2024-09-19T06:37:42.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 59562 7.09 MB/sec execute 328 sec latency 86.154 ms 2024-09-19T06:37:42.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:42 smithi053 ceph-mon[27968]: pgmap v846: 129 pgs: 129 active+clean; 437 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 5.5 KiB/s rd, 2.1 MiB/s wr, 163 op/s 2024-09-19T06:37:42.858 INFO:tasks.workunit.client.0.smithi053.stdout: 10 231343 27.56 MB/sec execute 326 sec latency 258.490 ms 2024-09-19T06:37:42.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:42 smithi155 ceph-mon[35292]: pgmap v846: 129 pgs: 129 active+clean; 437 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 5.5 KiB/s rd, 2.1 MiB/s wr, 163 op/s 2024-09-19T06:37:43.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 59795 7.14 MB/sec execute 329 sec latency 87.527 ms 2024-09-19T06:37:43.858 INFO:tasks.workunit.client.0.smithi053.stdout: 10 231532 27.49 MB/sec execute 327 sec latency 96.603 ms 2024-09-19T06:37:44.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 60029 7.16 MB/sec execute 330 sec latency 103.170 ms 2024-09-19T06:37:44.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:44 smithi053 ceph-mon[27968]: pgmap v847: 129 pgs: 129 active+clean; 448 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.0 KiB/s rd, 3.4 MiB/s wr, 207 op/s 2024-09-19T06:37:44.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:44 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:37:44.858 INFO:tasks.workunit.client.0.smithi053.stdout: 10 232543 27.56 MB/sec execute 328 sec latency 234.354 ms 2024-09-19T06:37:44.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:44 smithi155 ceph-mon[35292]: pgmap v847: 129 pgs: 129 active+clean; 448 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.0 KiB/s rd, 3.4 MiB/s wr, 207 op/s 2024-09-19T06:37:44.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:44 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:37:45.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 60158 7.17 MB/sec execute 331 sec latency 137.248 ms 2024-09-19T06:37:45.858 INFO:tasks.workunit.client.0.smithi053.stdout: 10 233567 27.68 MB/sec execute 329 sec latency 215.595 ms 2024-09-19T06:37:46.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 60208 7.15 MB/sec execute 332 sec latency 128.442 ms 2024-09-19T06:37:46.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:46 smithi053 ceph-mon[27968]: pgmap v848: 129 pgs: 129 active+clean; 451 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 3.1 KiB/s rd, 2.9 MiB/s wr, 138 op/s 2024-09-19T06:37:46.858 INFO:tasks.workunit.client.0.smithi053.stdout: 10 234832 27.81 MB/sec execute 330 sec latency 126.043 ms 2024-09-19T06:37:46.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:46 smithi155 ceph-mon[35292]: pgmap v848: 129 pgs: 129 active+clean; 451 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 3.1 KiB/s rd, 2.9 MiB/s wr, 138 op/s 2024-09-19T06:37:47.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 60291 7.15 MB/sec execute 333 sec latency 94.479 ms 2024-09-19T06:37:47.858 INFO:tasks.workunit.client.0.smithi053.stdout: 10 234951 27.73 MB/sec execute 331 sec latency 170.265 ms 2024-09-19T06:37:48.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 60454 7.17 MB/sec execute 334 sec latency 95.092 ms 2024-09-19T06:37:48.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:48 smithi053 ceph-mon[27968]: pgmap v849: 129 pgs: 129 active+clean; 459 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 3.1 KiB/s rd, 3.5 MiB/s wr, 136 op/s 2024-09-19T06:37:48.862 INFO:tasks.workunit.client.0.smithi053.stdout: 10 235183 27.67 MB/sec execute 332 sec latency 228.074 ms 2024-09-19T06:37:48.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:48 smithi155 ceph-mon[35292]: pgmap v849: 129 pgs: 129 active+clean; 459 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 3.1 KiB/s rd, 3.5 MiB/s wr, 136 op/s 2024-09-19T06:37:49.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 60609 7.18 MB/sec execute 335 sec latency 136.768 ms 2024-09-19T06:37:49.858 INFO:tasks.workunit.client.0.smithi053.stdout: 10 236552 27.76 MB/sec execute 333 sec latency 217.638 ms 2024-09-19T06:37:50.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 60737 7.18 MB/sec execute 336 sec latency 94.579 ms 2024-09-19T06:37:50.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:50 smithi053 ceph-mon[27968]: pgmap v850: 129 pgs: 129 active+clean; 464 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 5.1 KiB/s rd, 4.0 MiB/s wr, 134 op/s 2024-09-19T06:37:50.858 INFO:tasks.workunit.client.0.smithi053.stdout: 10 237472 27.90 MB/sec execute 334 sec latency 123.288 ms 2024-09-19T06:37:50.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:50 smithi155 ceph-mon[35292]: pgmap v850: 129 pgs: 129 active+clean; 464 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 5.1 KiB/s rd, 4.0 MiB/s wr, 134 op/s 2024-09-19T06:37:51.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 60847 7.17 MB/sec execute 337 sec latency 95.064 ms 2024-09-19T06:37:51.858 INFO:tasks.workunit.client.0.smithi053.stdout: 10 238432 27.97 MB/sec execute 335 sec latency 153.199 ms 2024-09-19T06:37:52.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 60954 7.15 MB/sec execute 338 sec latency 110.764 ms 2024-09-19T06:37:52.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:52 smithi053 ceph-mon[27968]: pgmap v851: 129 pgs: 129 active+clean; 464 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 3.5 KiB/s rd, 4.0 MiB/s wr, 115 op/s 2024-09-19T06:37:52.858 INFO:tasks.workunit.client.0.smithi053.stdout: 10 238575 27.89 MB/sec execute 336 sec latency 126.870 ms 2024-09-19T06:37:52.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:52 smithi155 ceph-mon[35292]: pgmap v851: 129 pgs: 129 active+clean; 464 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 3.5 KiB/s rd, 4.0 MiB/s wr, 115 op/s 2024-09-19T06:37:53.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 61084 7.14 MB/sec execute 339 sec latency 104.667 ms 2024-09-19T06:37:53.858 INFO:tasks.workunit.client.0.smithi053.stdout: 10 239164 27.93 MB/sec execute 337 sec latency 483.648 ms 2024-09-19T06:37:54.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 61210 7.13 MB/sec execute 340 sec latency 200.447 ms 2024-09-19T06:37:54.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:54 smithi053 ceph-mon[27968]: pgmap v852: 129 pgs: 129 active+clean; 443 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.2 KiB/s rd, 5.2 MiB/s wr, 162 op/s 2024-09-19T06:37:54.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:54 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:37:54.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:54 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:37:54.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:54 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:37:54.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:54 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:37:54.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:54 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:37:54.858 INFO:tasks.workunit.client.0.smithi053.stdout: 10 240334 27.99 MB/sec execute 338 sec latency 165.673 ms 2024-09-19T06:37:54.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:54 smithi155 ceph-mon[35292]: pgmap v852: 129 pgs: 129 active+clean; 443 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.2 KiB/s rd, 5.2 MiB/s wr, 162 op/s 2024-09-19T06:37:54.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:54 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:37:54.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:54 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:37:54.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:54 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:37:54.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:54 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:37:54.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:54 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:37:55.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 61349 7.12 MB/sec execute 341 sec latency 152.449 ms 2024-09-19T06:37:55.859 INFO:tasks.workunit.client.0.smithi053.stdout: 10 241166 28.08 MB/sec execute 339 sec latency 152.821 ms 2024-09-19T06:37:56.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 61499 7.12 MB/sec execute 342 sec latency 115.842 ms 2024-09-19T06:37:56.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:56 smithi053 ceph-mon[27968]: pgmap v853: 129 pgs: 129 active+clean; 443 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 5.7 KiB/s rd, 4.0 MiB/s wr, 121 op/s 2024-09-19T06:37:56.858 INFO:tasks.workunit.client.0.smithi053.stdout: 10 241980 28.12 MB/sec execute 340 sec latency 177.778 ms 2024-09-19T06:37:56.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:56 smithi155 ceph-mon[35292]: pgmap v853: 129 pgs: 129 active+clean; 443 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 5.7 KiB/s rd, 4.0 MiB/s wr, 121 op/s 2024-09-19T06:37:57.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 61662 7.12 MB/sec execute 343 sec latency 124.023 ms 2024-09-19T06:37:57.859 INFO:tasks.workunit.client.0.smithi053.stdout: 10 242202 28.05 MB/sec execute 341 sec latency 102.338 ms 2024-09-19T06:37:58.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 61853 7.15 MB/sec execute 344 sec latency 83.082 ms 2024-09-19T06:37:58.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:58 smithi053 ceph-mon[27968]: pgmap v854: 129 pgs: 129 active+clean; 433 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 5.7 KiB/s rd, 3.8 MiB/s wr, 127 op/s 2024-09-19T06:37:58.859 INFO:tasks.workunit.client.0.smithi053.stdout: 10 243178 28.10 MB/sec execute 342 sec latency 298.299 ms 2024-09-19T06:37:58.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:58 smithi155 ceph-mon[35292]: pgmap v854: 129 pgs: 129 active+clean; 433 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 5.7 KiB/s rd, 3.8 MiB/s wr, 127 op/s 2024-09-19T06:37:59.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 62012 7.15 MB/sec execute 345 sec latency 105.908 ms 2024-09-19T06:37:59.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:37:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:37:59.858 INFO:tasks.workunit.client.0.smithi053.stdout: 10 243637 28.07 MB/sec execute 343 sec latency 456.106 ms 2024-09-19T06:37:59.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:37:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:38:00.042 INFO:tasks.workunit.client.1.smithi155.stdout: 10 62138 7.13 MB/sec execute 346 sec latency 166.407 ms 2024-09-19T06:38:00.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:00 smithi053 ceph-mon[27968]: pgmap v855: 129 pgs: 129 active+clean; 427 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 10 KiB/s rd, 3.8 MiB/s wr, 187 op/s 2024-09-19T06:38:00.859 INFO:tasks.workunit.client.0.smithi053.stdout: 10 244126 28.13 MB/sec execute 344 sec latency 509.350 ms 2024-09-19T06:38:00.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:00 smithi155 ceph-mon[35292]: pgmap v855: 129 pgs: 129 active+clean; 427 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 10 KiB/s rd, 3.8 MiB/s wr, 187 op/s 2024-09-19T06:38:01.043 INFO:tasks.workunit.client.1.smithi155.stdout: 10 62302 7.12 MB/sec execute 347 sec latency 100.071 ms 2024-09-19T06:38:01.859 INFO:tasks.workunit.client.0.smithi053.stdout: 10 244431 28.09 MB/sec execute 345 sec latency 448.457 ms 2024-09-19T06:38:02.043 INFO:tasks.workunit.client.1.smithi155.stdout: 10 62444 7.12 MB/sec execute 348 sec latency 185.951 ms 2024-09-19T06:38:02.859 INFO:tasks.workunit.client.0.smithi053.stdout: 10 244563 28.04 MB/sec execute 346 sec latency 383.341 ms 2024-09-19T06:38:02.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:02 smithi155 ceph-mon[35292]: pgmap v856: 129 pgs: 129 active+clean; 425 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 8.0 KiB/s rd, 3.2 MiB/s wr, 165 op/s 2024-09-19T06:38:03.043 INFO:tasks.workunit.client.1.smithi155.stdout: 10 62600 7.11 MB/sec execute 349 sec latency 112.283 ms 2024-09-19T06:38:03.080 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:02 smithi053 ceph-mon[27968]: pgmap v856: 129 pgs: 129 active+clean; 425 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 8.0 KiB/s rd, 3.2 MiB/s wr, 165 op/s 2024-09-19T06:38:03.859 INFO:tasks.workunit.client.0.smithi053.stdout: 10 245376 28.08 MB/sec execute 347 sec latency 369.675 ms 2024-09-19T06:38:04.043 INFO:tasks.workunit.client.1.smithi155.stdout: 10 62783 7.12 MB/sec execute 350 sec latency 113.682 ms 2024-09-19T06:38:04.859 INFO:tasks.workunit.client.0.smithi053.stdout: 10 245481 28.02 MB/sec execute 348 sec latency 382.352 ms 2024-09-19T06:38:04.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:04 smithi155 ceph-mon[35292]: pgmap v857: 129 pgs: 129 active+clean; 428 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 9.7 KiB/s rd, 3.5 MiB/s wr, 216 op/s 2024-09-19T06:38:05.043 INFO:tasks.workunit.client.1.smithi155.stdout: 10 62894 7.11 MB/sec execute 351 sec latency 115.753 ms 2024-09-19T06:38:05.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:04 smithi053 ceph-mon[27968]: pgmap v857: 129 pgs: 129 active+clean; 428 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 9.7 KiB/s rd, 3.5 MiB/s wr, 216 op/s 2024-09-19T06:38:05.859 INFO:tasks.workunit.client.0.smithi053.stdout: 10 245517 27.94 MB/sec execute 349 sec latency 440.862 ms 2024-09-19T06:38:06.043 INFO:tasks.workunit.client.1.smithi155.stdout: 10 63000 7.11 MB/sec execute 352 sec latency 171.214 ms 2024-09-19T06:38:06.859 INFO:tasks.workunit.client.0.smithi053.stdout: 10 245541 27.86 MB/sec execute 350 sec latency 308.824 ms 2024-09-19T06:38:06.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:06 smithi155 ceph-mon[35292]: pgmap v858: 129 pgs: 129 active+clean; 429 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 6.1 KiB/s rd, 2.3 MiB/s wr, 174 op/s 2024-09-19T06:38:07.043 INFO:tasks.workunit.client.1.smithi155.stdout: 10 63127 7.13 MB/sec execute 353 sec latency 116.109 ms 2024-09-19T06:38:07.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:06 smithi053 ceph-mon[27968]: pgmap v858: 129 pgs: 129 active+clean; 429 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 6.1 KiB/s rd, 2.3 MiB/s wr, 174 op/s 2024-09-19T06:38:07.859 INFO:tasks.workunit.client.0.smithi053.stdout: 10 245592 27.78 MB/sec execute 351 sec latency 314.413 ms 2024-09-19T06:38:08.043 INFO:tasks.workunit.client.1.smithi155.stdout: 10 63328 7.18 MB/sec execute 354 sec latency 123.437 ms 2024-09-19T06:38:08.859 INFO:tasks.workunit.client.0.smithi053.stdout: 10 245637 27.71 MB/sec execute 352 sec latency 325.546 ms 2024-09-19T06:38:08.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:08 smithi155 ceph-mon[35292]: pgmap v859: 129 pgs: 129 active+clean; 432 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 6.1 KiB/s rd, 2.6 MiB/s wr, 176 op/s 2024-09-19T06:38:09.043 INFO:tasks.workunit.client.1.smithi155.stdout: 10 63506 7.19 MB/sec execute 355 sec latency 255.663 ms 2024-09-19T06:38:09.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:08 smithi053 ceph-mon[27968]: pgmap v859: 129 pgs: 129 active+clean; 432 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 6.1 KiB/s rd, 2.6 MiB/s wr, 176 op/s 2024-09-19T06:38:09.859 INFO:tasks.workunit.client.0.smithi053.stdout: 10 245721 27.64 MB/sec execute 353 sec latency 303.954 ms 2024-09-19T06:38:10.043 INFO:tasks.workunit.client.1.smithi155.stdout: 10 63666 7.20 MB/sec execute 356 sec latency 178.092 ms 2024-09-19T06:38:10.859 INFO:tasks.workunit.client.0.smithi053.stdout: 10 246359 27.67 MB/sec execute 354 sec latency 375.244 ms 2024-09-19T06:38:10.889 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:10 smithi155 ceph-mon[35292]: pgmap v860: 129 pgs: 129 active+clean; 442 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 7.7 KiB/s rd, 3.5 MiB/s wr, 197 op/s 2024-09-19T06:38:11.043 INFO:tasks.workunit.client.1.smithi155.stdout: 10 63732 7.19 MB/sec execute 357 sec latency 135.077 ms 2024-09-19T06:38:11.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:10 smithi053 ceph-mon[27968]: pgmap v860: 129 pgs: 129 active+clean; 442 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 7.7 KiB/s rd, 3.5 MiB/s wr, 197 op/s 2024-09-19T06:38:11.859 INFO:tasks.workunit.client.0.smithi053.stdout: 10 246770 27.62 MB/sec execute 355 sec latency 365.533 ms 2024-09-19T06:38:12.043 INFO:tasks.workunit.client.1.smithi155.stdout: 10 63769 7.17 MB/sec execute 358 sec latency 147.974 ms 2024-09-19T06:38:12.859 INFO:tasks.workunit.client.0.smithi053.stdout: 10 247549 27.69 MB/sec execute 356 sec latency 270.576 ms 2024-09-19T06:38:12.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:12 smithi155 ceph-mon[35292]: pgmap v861: 129 pgs: 129 active+clean; 443 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 3.4 KiB/s rd, 2.9 MiB/s wr, 123 op/s 2024-09-19T06:38:13.043 INFO:tasks.workunit.client.1.smithi155.stdout: 10 63875 7.18 MB/sec execute 359 sec latency 131.792 ms 2024-09-19T06:38:13.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:12 smithi053 ceph-mon[27968]: pgmap v861: 129 pgs: 129 active+clean; 443 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 3.4 KiB/s rd, 2.9 MiB/s wr, 123 op/s 2024-09-19T06:38:13.859 INFO:tasks.workunit.client.0.smithi053.stdout: 10 247980 27.68 MB/sec execute 357 sec latency 342.630 ms 2024-09-19T06:38:14.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 64027 7.19 MB/sec execute 360 sec latency 151.191 ms 2024-09-19T06:38:14.859 INFO:tasks.workunit.client.0.smithi053.stdout: 10 248296 27.66 MB/sec execute 358 sec latency 382.675 ms 2024-09-19T06:38:14.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:14 smithi155 ceph-mon[35292]: pgmap v862: 129 pgs: 129 active+clean; 453 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 5.3 KiB/s rd, 3.9 MiB/s wr, 151 op/s 2024-09-19T06:38:14.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:14 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:38:15.043 INFO:tasks.workunit.client.1.smithi155.stdout: 10 64169 7.20 MB/sec execute 361 sec latency 189.529 ms 2024-09-19T06:38:15.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:14 smithi053 ceph-mon[27968]: pgmap v862: 129 pgs: 129 active+clean; 453 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 5.3 KiB/s rd, 3.9 MiB/s wr, 151 op/s 2024-09-19T06:38:15.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:14 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:38:15.859 INFO:tasks.workunit.client.0.smithi053.stdout: 10 249012 27.69 MB/sec execute 359 sec latency 383.935 ms 2024-09-19T06:38:16.043 INFO:tasks.workunit.client.1.smithi155.stdout: 10 64283 7.19 MB/sec execute 362 sec latency 138.628 ms 2024-09-19T06:38:16.859 INFO:tasks.workunit.client.0.smithi053.stdout: 10 249058 27.62 MB/sec execute 360 sec latency 308.903 ms 2024-09-19T06:38:16.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:16 smithi155 ceph-mon[35292]: pgmap v863: 129 pgs: 129 active+clean; 454 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 3.6 KiB/s rd, 3.7 MiB/s wr, 101 op/s 2024-09-19T06:38:17.043 INFO:tasks.workunit.client.1.smithi155.stdout: 10 64364 7.18 MB/sec execute 363 sec latency 144.921 ms 2024-09-19T06:38:17.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:16 smithi053 ceph-mon[27968]: pgmap v863: 129 pgs: 129 active+clean; 454 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 3.6 KiB/s rd, 3.7 MiB/s wr, 101 op/s 2024-09-19T06:38:17.860 INFO:tasks.workunit.client.0.smithi053.stdout: 10 249070 27.54 MB/sec execute 361 sec latency 332.855 ms 2024-09-19T06:38:18.043 INFO:tasks.workunit.client.1.smithi155.stdout: 10 64437 7.16 MB/sec execute 364 sec latency 184.822 ms 2024-09-19T06:38:18.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:18 smithi053 ceph-mon[27968]: pgmap v864: 129 pgs: 129 active+clean; 455 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 3.6 KiB/s rd, 4.1 MiB/s wr, 98 op/s 2024-09-19T06:38:18.861 INFO:tasks.workunit.client.0.smithi053.stdout: 10 249128 27.47 MB/sec execute 362 sec latency 287.979 ms 2024-09-19T06:38:18.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:18 smithi155 ceph-mon[35292]: pgmap v864: 129 pgs: 129 active+clean; 455 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 3.6 KiB/s rd, 4.1 MiB/s wr, 98 op/s 2024-09-19T06:38:19.044 INFO:tasks.workunit.client.1.smithi155.stdout: 10 64517 7.15 MB/sec execute 365 sec latency 188.121 ms 2024-09-19T06:38:19.860 INFO:tasks.workunit.client.0.smithi053.stdout: 10 249157 27.39 MB/sec execute 363 sec latency 283.983 ms 2024-09-19T06:38:20.043 INFO:tasks.workunit.client.1.smithi155.stdout: 10 64595 7.14 MB/sec execute 366 sec latency 328.673 ms 2024-09-19T06:38:20.860 INFO:tasks.workunit.client.0.smithi053.stdout: 10 249315 27.33 MB/sec execute 364 sec latency 297.260 ms 2024-09-19T06:38:20.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:20 smithi155 ceph-mon[35292]: pgmap v865: 129 pgs: 129 active+clean; 458 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 6.6 KiB/s rd, 4.4 MiB/s wr, 133 op/s 2024-09-19T06:38:21.043 INFO:tasks.workunit.client.1.smithi155.stdout: 10 64721 7.12 MB/sec execute 367 sec latency 227.110 ms 2024-09-19T06:38:21.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:20 smithi053 ceph-mon[27968]: pgmap v865: 129 pgs: 129 active+clean; 458 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 6.6 KiB/s rd, 4.4 MiB/s wr, 133 op/s 2024-09-19T06:38:21.860 INFO:tasks.workunit.client.0.smithi053.stdout: 10 250654 27.42 MB/sec execute 365 sec latency 313.575 ms 2024-09-19T06:38:22.043 INFO:tasks.workunit.client.1.smithi155.stdout: 10 64868 7.12 MB/sec execute 368 sec latency 127.345 ms 2024-09-19T06:38:22.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:22 smithi155 ceph-mon[35292]: pgmap v866: 129 pgs: 129 active+clean; 456 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 4.9 KiB/s rd, 3.5 MiB/s wr, 101 op/s 2024-09-19T06:38:22.860 INFO:tasks.workunit.client.0.smithi053.stdout: 10 251605 27.54 MB/sec execute 366 sec latency 123.926 ms 2024-09-19T06:38:23.043 INFO:tasks.workunit.client.1.smithi155.stdout: 10 65013 7.12 MB/sec execute 369 sec latency 91.711 ms 2024-09-19T06:38:23.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:22 smithi053 ceph-mon[27968]: pgmap v866: 129 pgs: 129 active+clean; 456 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 4.9 KiB/s rd, 3.5 MiB/s wr, 101 op/s 2024-09-19T06:38:23.860 INFO:tasks.workunit.client.0.smithi053.stdout: 10 252623 27.61 MB/sec execute 367 sec latency 117.299 ms 2024-09-19T06:38:24.043 INFO:tasks.workunit.client.1.smithi155.stdout: 10 65162 7.11 MB/sec execute 370 sec latency 139.918 ms 2024-09-19T06:38:24.860 INFO:tasks.workunit.client.0.smithi053.stdout: 10 253261 27.64 MB/sec execute 368 sec latency 94.635 ms 2024-09-19T06:38:24.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:24 smithi155 ceph-mon[35292]: pgmap v867: 129 pgs: 129 active+clean; 449 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 9.1 KiB/s rd, 3.7 MiB/s wr, 150 op/s 2024-09-19T06:38:25.043 INFO:tasks.workunit.client.1.smithi155.stdout: 10 65346 7.14 MB/sec execute 371 sec latency 101.142 ms 2024-09-19T06:38:25.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:24 smithi053 ceph-mon[27968]: pgmap v867: 129 pgs: 129 active+clean; 449 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 9.1 KiB/s rd, 3.7 MiB/s wr, 150 op/s 2024-09-19T06:38:25.860 INFO:tasks.workunit.client.0.smithi053.stdout: 10 254574 27.73 MB/sec execute 369 sec latency 117.185 ms 2024-09-19T06:38:26.044 INFO:tasks.workunit.client.1.smithi155.stdout: 10 65522 7.14 MB/sec execute 372 sec latency 131.608 ms 2024-09-19T06:38:26.860 INFO:tasks.workunit.client.0.smithi053.stdout: 10 255025 27.73 MB/sec execute 370 sec latency 485.216 ms 2024-09-19T06:38:26.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:26 smithi155 ceph-mon[35292]: pgmap v868: 129 pgs: 129 active+clean; 446 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.2 KiB/s rd, 2.9 MiB/s wr, 138 op/s 2024-09-19T06:38:27.043 INFO:tasks.workunit.client.1.smithi155.stdout: 10 65613 7.13 MB/sec execute 373 sec latency 183.925 ms 2024-09-19T06:38:27.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:26 smithi053 ceph-mon[27968]: pgmap v868: 129 pgs: 129 active+clean; 446 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.2 KiB/s rd, 2.9 MiB/s wr, 138 op/s 2024-09-19T06:38:27.860 INFO:tasks.workunit.client.0.smithi053.stdout: 10 255148 27.69 MB/sec execute 371 sec latency 539.432 ms 2024-09-19T06:38:28.044 INFO:tasks.workunit.client.1.smithi155.stdout: 10 65761 7.12 MB/sec execute 374 sec latency 199.912 ms 2024-09-19T06:38:28.860 INFO:tasks.workunit.client.0.smithi053.stdout: 10 255213 27.62 MB/sec execute 372 sec latency 457.556 ms 2024-09-19T06:38:28.906 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:28 smithi053 ceph-mon[27968]: pgmap v869: 129 pgs: 129 active+clean; 440 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 7.2 KiB/s rd, 3.0 MiB/s wr, 163 op/s 2024-09-19T06:38:28.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:28 smithi155 ceph-mon[35292]: pgmap v869: 129 pgs: 129 active+clean; 440 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 7.2 KiB/s rd, 3.0 MiB/s wr, 163 op/s 2024-09-19T06:38:29.043 INFO:tasks.workunit.client.1.smithi155.stdout: 10 65872 7.10 MB/sec execute 375 sec latency 207.949 ms 2024-09-19T06:38:29.860 INFO:tasks.workunit.client.0.smithi053.stdout: 10 256012 27.66 MB/sec execute 373 sec latency 466.550 ms 2024-09-19T06:38:29.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:29 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:38:30.044 INFO:tasks.workunit.client.1.smithi155.stdout: 10 66041 7.10 MB/sec execute 376 sec latency 155.533 ms 2024-09-19T06:38:30.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:38:30.860 INFO:tasks.workunit.client.0.smithi053.stdout: 10 256107 27.60 MB/sec execute 374 sec latency 450.423 ms 2024-09-19T06:38:30.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:30 smithi155 ceph-mon[35292]: pgmap v870: 129 pgs: 129 active+clean; 440 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 9.2 KiB/s rd, 2.7 MiB/s wr, 175 op/s 2024-09-19T06:38:31.044 INFO:tasks.workunit.client.1.smithi155.stdout: 10 66147 7.09 MB/sec execute 377 sec latency 205.186 ms 2024-09-19T06:38:31.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:30 smithi053 ceph-mon[27968]: pgmap v870: 129 pgs: 129 active+clean; 440 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 9.2 KiB/s rd, 2.7 MiB/s wr, 175 op/s 2024-09-19T06:38:31.860 INFO:tasks.workunit.client.0.smithi053.stdout: 10 256152 27.53 MB/sec execute 375 sec latency 483.914 ms 2024-09-19T06:38:32.044 INFO:tasks.workunit.client.1.smithi155.stdout: 10 66319 7.10 MB/sec execute 378 sec latency 153.887 ms 2024-09-19T06:38:32.860 INFO:tasks.workunit.client.0.smithi053.stdout: 10 256161 27.46 MB/sec execute 376 sec latency 474.825 ms 2024-09-19T06:38:32.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:32 smithi155 ceph-mon[35292]: pgmap v871: 129 pgs: 129 active+clean; 442 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 6.2 KiB/s rd, 2.2 MiB/s wr, 161 op/s 2024-09-19T06:38:33.044 INFO:tasks.workunit.client.1.smithi155.stdout: 10 66424 7.09 MB/sec execute 379 sec latency 137.461 ms 2024-09-19T06:38:33.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:32 smithi053 ceph-mon[27968]: pgmap v871: 129 pgs: 129 active+clean; 442 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 6.2 KiB/s rd, 2.2 MiB/s wr, 161 op/s 2024-09-19T06:38:33.860 INFO:tasks.workunit.client.0.smithi053.stdout: 10 256210 27.39 MB/sec execute 377 sec latency 366.935 ms 2024-09-19T06:38:34.044 INFO:tasks.workunit.client.1.smithi155.stdout: 10 66495 7.09 MB/sec execute 380 sec latency 208.930 ms 2024-09-19T06:38:34.860 INFO:tasks.workunit.client.0.smithi053.stdout: 10 256221 27.32 MB/sec execute 378 sec latency 516.093 ms 2024-09-19T06:38:34.920 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:34 smithi053 ceph-mon[27968]: pgmap v872: 129 pgs: 129 active+clean; 441 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.8 KiB/s rd, 2.5 MiB/s wr, 187 op/s 2024-09-19T06:38:34.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:34 smithi155 ceph-mon[35292]: pgmap v872: 129 pgs: 129 active+clean; 441 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.8 KiB/s rd, 2.5 MiB/s wr, 187 op/s 2024-09-19T06:38:35.044 INFO:tasks.workunit.client.1.smithi155.stdout: 10 66588 7.08 MB/sec execute 381 sec latency 177.820 ms 2024-09-19T06:38:35.860 INFO:tasks.workunit.client.0.smithi053.stdout: 10 256231 27.25 MB/sec execute 379 sec latency 502.580 ms 2024-09-19T06:38:36.044 INFO:tasks.workunit.client.1.smithi155.stdout: 10 66738 7.11 MB/sec execute 382 sec latency 303.740 ms 2024-09-19T06:38:36.860 INFO:tasks.workunit.client.0.smithi053.stdout: 10 256299 27.18 MB/sec execute 380 sec latency 479.589 ms 2024-09-19T06:38:36.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:36 smithi155 ceph-mon[35292]: pgmap v873: 129 pgs: 129 active+clean; 444 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 3.7 KiB/s rd, 2.4 MiB/s wr, 153 op/s 2024-09-19T06:38:37.044 INFO:tasks.workunit.client.1.smithi155.stdout: 10 66899 7.14 MB/sec execute 383 sec latency 147.578 ms 2024-09-19T06:38:37.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:36 smithi053 ceph-mon[27968]: pgmap v873: 129 pgs: 129 active+clean; 444 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 3.7 KiB/s rd, 2.4 MiB/s wr, 153 op/s 2024-09-19T06:38:37.860 INFO:tasks.workunit.client.0.smithi053.stdout: 10 256347 27.11 MB/sec execute 381 sec latency 417.450 ms 2024-09-19T06:38:38.044 INFO:tasks.workunit.client.1.smithi155.stdout: 10 67086 7.15 MB/sec execute 384 sec latency 165.556 ms 2024-09-19T06:38:38.860 INFO:tasks.workunit.client.0.smithi053.stdout: 10 256753 27.12 MB/sec execute 382 sec latency 444.925 ms 2024-09-19T06:38:38.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:38 smithi155 ceph-mon[35292]: pgmap v874: 129 pgs: 129 active+clean; 439 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 3.7 KiB/s rd, 2.5 MiB/s wr, 135 op/s 2024-09-19T06:38:39.044 INFO:tasks.workunit.client.1.smithi155.stdout: 10 67218 7.15 MB/sec execute 385 sec latency 162.144 ms 2024-09-19T06:38:39.048 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:38 smithi053 ceph-mon[27968]: pgmap v874: 129 pgs: 129 active+clean; 439 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 3.7 KiB/s rd, 2.5 MiB/s wr, 135 op/s 2024-09-19T06:38:39.860 INFO:tasks.workunit.client.0.smithi053.stdout: 10 256996 27.07 MB/sec execute 383 sec latency 589.777 ms 2024-09-19T06:38:40.044 INFO:tasks.workunit.client.1.smithi155.stdout: 10 67274 7.14 MB/sec execute 386 sec latency 184.053 ms 2024-09-19T06:38:40.860 INFO:tasks.workunit.client.0.smithi053.stdout: 10 257287 27.02 MB/sec execute 384 sec latency 493.060 ms 2024-09-19T06:38:40.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:40 smithi155 ceph-mon[35292]: pgmap v875: 129 pgs: 129 active+clean; 435 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 5.2 KiB/s rd, 3.4 MiB/s wr, 139 op/s 2024-09-19T06:38:41.044 INFO:tasks.workunit.client.1.smithi155.stdout: 10 67312 7.13 MB/sec execute 387 sec latency 148.885 ms 2024-09-19T06:38:41.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:40 smithi053 ceph-mon[27968]: pgmap v875: 129 pgs: 129 active+clean; 435 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 5.2 KiB/s rd, 3.4 MiB/s wr, 139 op/s 2024-09-19T06:38:41.861 INFO:tasks.workunit.client.0.smithi053.stdout: 10 257687 26.98 MB/sec execute 385 sec latency 566.072 ms 2024-09-19T06:38:42.044 INFO:tasks.workunit.client.1.smithi155.stdout: 10 67363 7.11 MB/sec execute 388 sec latency 143.631 ms 2024-09-19T06:38:42.861 INFO:tasks.workunit.client.0.smithi053.stdout: 10 257825 26.93 MB/sec execute 386 sec latency 505.466 ms 2024-09-19T06:38:42.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:42 smithi155 ceph-mon[35292]: pgmap v876: 129 pgs: 129 active+clean; 437 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 3.2 MiB/s wr, 122 op/s 2024-09-19T06:38:43.044 INFO:tasks.workunit.client.1.smithi155.stdout: 10 67478 7.13 MB/sec execute 389 sec latency 154.767 ms 2024-09-19T06:38:43.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:42 smithi053 ceph-mon[27968]: pgmap v876: 129 pgs: 129 active+clean; 437 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 3.2 MiB/s wr, 122 op/s 2024-09-19T06:38:43.861 INFO:tasks.workunit.client.0.smithi053.stdout: 10 258233 26.96 MB/sec execute 387 sec latency 568.860 ms 2024-09-19T06:38:44.044 INFO:tasks.workunit.client.1.smithi155.stdout: 10 67636 7.14 MB/sec execute 390 sec latency 165.239 ms 2024-09-19T06:38:44.861 INFO:tasks.workunit.client.0.smithi053.stdout: 10 258471 26.93 MB/sec execute 388 sec latency 557.491 ms 2024-09-19T06:38:44.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:44 smithi155 ceph-mon[35292]: pgmap v877: 129 pgs: 129 active+clean; 441 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 4.8 KiB/s rd, 3.6 MiB/s wr, 120 op/s 2024-09-19T06:38:44.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:44 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:38:45.044 INFO:tasks.workunit.client.1.smithi155.stdout: 10 67760 7.14 MB/sec execute 391 sec latency 165.497 ms 2024-09-19T06:38:45.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:44 smithi053 ceph-mon[27968]: pgmap v877: 129 pgs: 129 active+clean; 441 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 4.8 KiB/s rd, 3.6 MiB/s wr, 120 op/s 2024-09-19T06:38:45.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:44 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:38:45.861 INFO:tasks.workunit.client.0.smithi053.stdout: 10 258671 26.89 MB/sec execute 389 sec latency 381.031 ms 2024-09-19T06:38:46.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 67843 7.13 MB/sec execute 392 sec latency 173.542 ms 2024-09-19T06:38:46.861 INFO:tasks.workunit.client.0.smithi053.stdout: 10 259376 26.92 MB/sec execute 390 sec latency 471.093 ms 2024-09-19T06:38:46.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:46 smithi155 ceph-mon[35292]: pgmap v878: 129 pgs: 129 active+clean; 442 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 3.5 MiB/s wr, 97 op/s 2024-09-19T06:38:47.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 67936 7.12 MB/sec execute 393 sec latency 213.247 ms 2024-09-19T06:38:47.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:46 smithi053 ceph-mon[27968]: pgmap v878: 129 pgs: 129 active+clean; 442 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 3.5 MiB/s wr, 97 op/s 2024-09-19T06:38:47.861 INFO:tasks.workunit.client.0.smithi053.stdout: 10 259651 26.89 MB/sec execute 391 sec latency 520.907 ms 2024-09-19T06:38:48.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 68026 7.11 MB/sec execute 394 sec latency 136.217 ms 2024-09-19T06:38:48.861 INFO:tasks.workunit.client.0.smithi053.stdout: 10 259706 26.83 MB/sec execute 392 sec latency 312.886 ms 2024-09-19T06:38:48.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:48 smithi155 ceph-mon[35292]: pgmap v879: 129 pgs: 129 active+clean; 442 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 3.6 MiB/s wr, 84 op/s 2024-09-19T06:38:49.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 68153 7.10 MB/sec execute 395 sec latency 149.662 ms 2024-09-19T06:38:49.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:48 smithi053 ceph-mon[27968]: pgmap v879: 129 pgs: 129 active+clean; 442 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 3.6 MiB/s wr, 84 op/s 2024-09-19T06:38:49.861 INFO:tasks.workunit.client.0.smithi053.stdout: 10 259955 26.77 MB/sec execute 393 sec latency 87.125 ms 2024-09-19T06:38:50.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 68290 7.09 MB/sec execute 396 sec latency 116.410 ms 2024-09-19T06:38:50.861 INFO:tasks.workunit.client.0.smithi053.stdout: 10 260837 26.82 MB/sec execute 394 sec latency 453.304 ms 2024-09-19T06:38:50.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:50 smithi155 ceph-mon[35292]: pgmap v880: 129 pgs: 129 active+clean; 445 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 5.8 KiB/s rd, 3.9 MiB/s wr, 119 op/s 2024-09-19T06:38:51.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 68435 7.08 MB/sec execute 397 sec latency 135.655 ms 2024-09-19T06:38:51.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:50 smithi053 ceph-mon[27968]: pgmap v880: 129 pgs: 129 active+clean; 445 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 5.8 KiB/s rd, 3.9 MiB/s wr, 119 op/s 2024-09-19T06:38:51.861 INFO:tasks.workunit.client.0.smithi053.stdout: 10 262121 26.95 MB/sec execute 395 sec latency 343.065 ms 2024-09-19T06:38:52.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 68582 7.08 MB/sec execute 398 sec latency 121.472 ms 2024-09-19T06:38:52.861 INFO:tasks.workunit.client.0.smithi053.stdout: 10 263244 27.04 MB/sec execute 396 sec latency 116.038 ms 2024-09-19T06:38:52.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:52 smithi155 ceph-mon[35292]: pgmap v881: 129 pgs: 129 active+clean; 444 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 4.2 KiB/s rd, 2.8 MiB/s wr, 88 op/s 2024-09-19T06:38:53.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 68747 7.08 MB/sec execute 399 sec latency 85.039 ms 2024-09-19T06:38:53.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:52 smithi053 ceph-mon[27968]: pgmap v881: 129 pgs: 129 active+clean; 444 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 4.2 KiB/s rd, 2.8 MiB/s wr, 88 op/s 2024-09-19T06:38:53.861 INFO:tasks.workunit.client.0.smithi053.stdout: 10 263431 26.98 MB/sec execute 397 sec latency 93.037 ms 2024-09-19T06:38:54.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 68950 7.11 MB/sec execute 400 sec latency 101.626 ms 2024-09-19T06:38:54.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:54 smithi053 ceph-mon[27968]: pgmap v882: 129 pgs: 129 active+clean; 432 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 9.0 KiB/s rd, 3.4 MiB/s wr, 160 op/s 2024-09-19T06:38:54.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:54 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:38:54.861 INFO:tasks.workunit.client.0.smithi053.stdout: 10 264866 27.07 MB/sec execute 398 sec latency 110.091 ms 2024-09-19T06:38:54.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:54 smithi155 ceph-mon[35292]: pgmap v882: 129 pgs: 129 active+clean; 432 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 9.0 KiB/s rd, 3.4 MiB/s wr, 160 op/s 2024-09-19T06:38:54.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:54 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:38:55.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 69098 7.11 MB/sec execute 401 sec latency 97.706 ms 2024-09-19T06:38:55.861 INFO:tasks.workunit.client.0.smithi053.stdout: 10 265839 27.19 MB/sec execute 399 sec latency 119.399 ms 2024-09-19T06:38:55.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:55 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:38:55.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:55 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:38:55.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:55 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:38:56.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 69255 7.10 MB/sec execute 402 sec latency 121.822 ms 2024-09-19T06:38:56.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:55 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:38:56.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:55 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:38:56.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:55 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:38:56.861 INFO:tasks.workunit.client.0.smithi053.stdout: 10 266558 27.22 MB/sec execute 400 sec latency 476.424 ms 2024-09-19T06:38:56.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:56 smithi155 ceph-mon[35292]: pgmap v883: 129 pgs: 129 active+clean; 427 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.3 KiB/s rd, 3.1 MiB/s wr, 150 op/s 2024-09-19T06:38:57.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 69438 7.09 MB/sec execute 403 sec latency 159.901 ms 2024-09-19T06:38:57.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:56 smithi053 ceph-mon[27968]: pgmap v883: 129 pgs: 129 active+clean; 427 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.3 KiB/s rd, 3.1 MiB/s wr, 150 op/s 2024-09-19T06:38:57.861 INFO:tasks.workunit.client.0.smithi053.stdout: 10 266742 27.17 MB/sec execute 401 sec latency 424.444 ms 2024-09-19T06:38:58.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 69570 7.08 MB/sec execute 404 sec latency 212.428 ms 2024-09-19T06:38:58.861 INFO:tasks.workunit.client.0.smithi053.stdout: 10 266778 27.11 MB/sec execute 402 sec latency 480.136 ms 2024-09-19T06:38:58.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:58 smithi155 ceph-mon[35292]: pgmap v884: 129 pgs: 129 active+clean; 428 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.3 KiB/s rd, 3.1 MiB/s wr, 159 op/s 2024-09-19T06:38:59.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 69697 7.08 MB/sec execute 405 sec latency 172.019 ms 2024-09-19T06:38:59.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:58 smithi053 ceph-mon[27968]: pgmap v884: 129 pgs: 129 active+clean; 428 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.3 KiB/s rd, 3.1 MiB/s wr, 159 op/s 2024-09-19T06:38:59.861 INFO:tasks.workunit.client.0.smithi053.stdout: 10 266797 27.04 MB/sec execute 403 sec latency 417.124 ms 2024-09-19T06:38:59.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:38:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:39:00.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 69841 7.07 MB/sec execute 406 sec latency 162.726 ms 2024-09-19T06:39:00.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:38:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:39:00.861 INFO:tasks.workunit.client.0.smithi053.stdout: 10 266803 26.98 MB/sec execute 404 sec latency 395.048 ms 2024-09-19T06:39:00.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:00 smithi155 ceph-mon[35292]: pgmap v885: 129 pgs: 129 active+clean; 431 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 9.2 KiB/s rd, 3.1 MiB/s wr, 206 op/s 2024-09-19T06:39:01.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 69941 7.07 MB/sec execute 407 sec latency 183.266 ms 2024-09-19T06:39:01.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:00 smithi053 ceph-mon[27968]: pgmap v885: 129 pgs: 129 active+clean; 431 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 9.2 KiB/s rd, 3.1 MiB/s wr, 206 op/s 2024-09-19T06:39:01.861 INFO:tasks.workunit.client.0.smithi053.stdout: 10 266850 26.91 MB/sec execute 405 sec latency 554.288 ms 2024-09-19T06:39:02.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 70014 7.06 MB/sec execute 408 sec latency 235.151 ms 2024-09-19T06:39:02.862 INFO:tasks.workunit.client.0.smithi053.stdout: 10 266867 26.85 MB/sec execute 406 sec latency 405.564 ms 2024-09-19T06:39:02.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:02 smithi155 ceph-mon[35292]: pgmap v886: 129 pgs: 129 active+clean; 432 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 6.7 KiB/s rd, 2.3 MiB/s wr, 166 op/s 2024-09-19T06:39:03.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 70087 7.06 MB/sec execute 409 sec latency 175.249 ms 2024-09-19T06:39:03.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:02 smithi053 ceph-mon[27968]: pgmap v886: 129 pgs: 129 active+clean; 432 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 6.7 KiB/s rd, 2.3 MiB/s wr, 166 op/s 2024-09-19T06:39:03.862 INFO:tasks.workunit.client.0.smithi053.stdout: 10 267553 26.87 MB/sec execute 407 sec latency 450.687 ms 2024-09-19T06:39:04.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 70226 7.07 MB/sec execute 410 sec latency 126.253 ms 2024-09-19T06:39:04.861 INFO:tasks.workunit.client.0.smithi053.stdout: 10 268864 26.98 MB/sec execute 408 sec latency 135.071 ms 2024-09-19T06:39:04.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:04 smithi155 ceph-mon[35292]: pgmap v887: 129 pgs: 129 active+clean; 437 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 8.1 KiB/s rd, 3.1 MiB/s wr, 203 op/s 2024-09-19T06:39:05.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 70466 7.12 MB/sec execute 411 sec latency 97.819 ms 2024-09-19T06:39:05.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:04 smithi053 ceph-mon[27968]: pgmap v887: 129 pgs: 129 active+clean; 437 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 8.1 KiB/s rd, 3.1 MiB/s wr, 203 op/s 2024-09-19T06:39:05.862 INFO:tasks.workunit.client.0.smithi053.stdout: 10 270283 27.11 MB/sec execute 409 sec latency 108.843 ms 2024-09-19T06:39:06.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 70708 7.14 MB/sec execute 412 sec latency 100.393 ms 2024-09-19T06:39:06.862 INFO:tasks.workunit.client.0.smithi053.stdout: 10 270407 27.05 MB/sec execute 410 sec latency 176.353 ms 2024-09-19T06:39:06.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:06 smithi155 ceph-mon[35292]: pgmap v888: 129 pgs: 129 active+clean; 439 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.7 MiB/s wr, 135 op/s 2024-09-19T06:39:07.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 70826 7.14 MB/sec execute 413 sec latency 118.368 ms 2024-09-19T06:39:07.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:06 smithi053 ceph-mon[27968]: pgmap v888: 129 pgs: 129 active+clean; 439 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.7 MiB/s wr, 135 op/s 2024-09-19T06:39:07.862 INFO:tasks.workunit.client.0.smithi053.stdout: 10 271442 27.10 MB/sec execute 411 sec latency 109.413 ms 2024-09-19T06:39:08.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 70868 7.12 MB/sec execute 414 sec latency 96.486 ms 2024-09-19T06:39:08.862 INFO:tasks.workunit.client.0.smithi053.stdout: 10 272444 27.19 MB/sec execute 412 sec latency 254.528 ms 2024-09-19T06:39:08.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:08 smithi155 ceph-mon[35292]: pgmap v889: 129 pgs: 129 active+clean; 444 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.8 MiB/s wr, 140 op/s 2024-09-19T06:39:09.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 70945 7.12 MB/sec execute 415 sec latency 181.088 ms 2024-09-19T06:39:09.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:08 smithi053 ceph-mon[27968]: pgmap v889: 129 pgs: 129 active+clean; 444 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.8 MiB/s wr, 140 op/s 2024-09-19T06:39:09.862 INFO:tasks.workunit.client.0.smithi053.stdout: 10 272742 27.16 MB/sec execute 413 sec latency 563.885 ms 2024-09-19T06:39:10.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 71087 7.14 MB/sec execute 416 sec latency 169.671 ms 2024-09-19T06:39:10.862 INFO:tasks.workunit.client.0.smithi053.stdout: 10 272885 27.12 MB/sec execute 414 sec latency 510.983 ms 2024-09-19T06:39:10.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:10 smithi155 ceph-mon[35292]: pgmap v890: 129 pgs: 129 active+clean; 453 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 5.2 KiB/s rd, 3.5 MiB/s wr, 142 op/s 2024-09-19T06:39:11.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 71212 7.14 MB/sec execute 417 sec latency 181.073 ms 2024-09-19T06:39:11.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:10 smithi053 ceph-mon[27968]: pgmap v890: 129 pgs: 129 active+clean; 453 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 5.2 KiB/s rd, 3.5 MiB/s wr, 142 op/s 2024-09-19T06:39:11.862 INFO:tasks.workunit.client.0.smithi053.stdout: 10 273021 27.07 MB/sec execute 415 sec latency 450.448 ms 2024-09-19T06:39:12.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 71326 7.14 MB/sec execute 418 sec latency 187.896 ms 2024-09-19T06:39:12.862 INFO:tasks.workunit.client.0.smithi053.stdout: 10 273654 27.09 MB/sec execute 416 sec latency 540.115 ms 2024-09-19T06:39:12.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:12 smithi155 ceph-mon[35292]: pgmap v891: 129 pgs: 129 active+clean; 455 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.3 KiB/s rd, 3.3 MiB/s wr, 90 op/s 2024-09-19T06:39:13.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 71413 7.13 MB/sec execute 419 sec latency 212.038 ms 2024-09-19T06:39:13.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:12 smithi053 ceph-mon[27968]: pgmap v891: 129 pgs: 129 active+clean; 455 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.3 KiB/s rd, 3.3 MiB/s wr, 90 op/s 2024-09-19T06:39:13.869 INFO:tasks.workunit.client.0.smithi053.stdout: 10 273838 27.05 MB/sec execute 417 sec latency 492.381 ms 2024-09-19T06:39:14.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 71485 7.12 MB/sec execute 420 sec latency 211.571 ms 2024-09-19T06:39:14.862 INFO:tasks.workunit.client.0.smithi053.stdout: 10 273957 26.99 MB/sec execute 418 sec latency 459.301 ms 2024-09-19T06:39:14.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:14 smithi155 ceph-mon[35292]: pgmap v892: 129 pgs: 129 active+clean; 457 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 5.7 KiB/s rd, 4.4 MiB/s wr, 135 op/s 2024-09-19T06:39:14.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:14 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:39:15.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 71583 7.10 MB/sec execute 421 sec latency 205.456 ms 2024-09-19T06:39:15.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:14 smithi053 ceph-mon[27968]: pgmap v892: 129 pgs: 129 active+clean; 457 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 5.7 KiB/s rd, 4.4 MiB/s wr, 135 op/s 2024-09-19T06:39:15.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:14 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:39:15.862 INFO:tasks.workunit.client.0.smithi053.stdout: 10 274961 27.04 MB/sec execute 419 sec latency 112.299 ms 2024-09-19T06:39:16.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 71711 7.10 MB/sec execute 422 sec latency 119.198 ms 2024-09-19T06:39:16.862 INFO:tasks.workunit.client.0.smithi053.stdout: 10 276073 27.13 MB/sec execute 420 sec latency 148.962 ms 2024-09-19T06:39:16.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:16 smithi155 ceph-mon[35292]: pgmap v893: 129 pgs: 129 active+clean; 457 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 4.3 KiB/s rd, 3.9 MiB/s wr, 104 op/s 2024-09-19T06:39:17.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 71859 7.09 MB/sec execute 423 sec latency 125.869 ms 2024-09-19T06:39:17.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:16 smithi053 ceph-mon[27968]: pgmap v893: 129 pgs: 129 active+clean; 457 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 4.3 KiB/s rd, 3.9 MiB/s wr, 104 op/s 2024-09-19T06:39:17.862 INFO:tasks.workunit.client.0.smithi053.stdout: 10 276468 27.13 MB/sec execute 421 sec latency 466.431 ms 2024-09-19T06:39:18.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 71967 7.08 MB/sec execute 424 sec latency 195.585 ms 2024-09-19T06:39:18.862 INFO:tasks.workunit.client.0.smithi053.stdout: 10 277436 27.18 MB/sec execute 422 sec latency 478.371 ms 2024-09-19T06:39:18.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:18 smithi155 ceph-mon[35292]: pgmap v894: 129 pgs: 129 active+clean; 457 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 4.3 KiB/s rd, 3.7 MiB/s wr, 102 op/s 2024-09-19T06:39:19.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 72118 7.08 MB/sec execute 425 sec latency 100.474 ms 2024-09-19T06:39:19.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:18 smithi053 ceph-mon[27968]: pgmap v894: 129 pgs: 129 active+clean; 457 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 4.3 KiB/s rd, 3.7 MiB/s wr, 102 op/s 2024-09-19T06:39:19.862 INFO:tasks.workunit.client.0.smithi053.stdout: 10 277569 27.13 MB/sec execute 423 sec latency 307.188 ms 2024-09-19T06:39:20.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 72259 7.08 MB/sec execute 426 sec latency 174.806 ms 2024-09-19T06:39:20.862 INFO:tasks.workunit.client.0.smithi053.stdout: 10 277662 27.07 MB/sec execute 424 sec latency 367.458 ms 2024-09-19T06:39:20.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:20 smithi155 ceph-mon[35292]: pgmap v895: 129 pgs: 129 active+clean; 434 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 9.8 KiB/s rd, 3.8 MiB/s wr, 133 op/s 2024-09-19T06:39:21.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 72386 7.08 MB/sec execute 427 sec latency 230.615 ms 2024-09-19T06:39:21.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:20 smithi053 ceph-mon[27968]: pgmap v895: 129 pgs: 129 active+clean; 434 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 9.8 KiB/s rd, 3.8 MiB/s wr, 133 op/s 2024-09-19T06:39:21.862 INFO:tasks.workunit.client.0.smithi053.stdout: 10 278075 27.08 MB/sec execute 425 sec latency 472.107 ms 2024-09-19T06:39:22.045 INFO:tasks.workunit.client.1.smithi155.stdout: 10 72533 7.09 MB/sec execute 428 sec latency 221.647 ms 2024-09-19T06:39:22.862 INFO:tasks.workunit.client.0.smithi053.stdout: 10 278352 27.04 MB/sec execute 426 sec latency 624.368 ms 2024-09-19T06:39:22.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:22 smithi155 ceph-mon[35292]: pgmap v896: 129 pgs: 129 active+clean; 429 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 7.9 KiB/s rd, 2.9 MiB/s wr, 129 op/s 2024-09-19T06:39:23.046 INFO:tasks.workunit.client.1.smithi155.stdout: 10 72665 7.09 MB/sec execute 429 sec latency 150.664 ms 2024-09-19T06:39:23.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:22 smithi053 ceph-mon[27968]: pgmap v896: 129 pgs: 129 active+clean; 429 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 7.9 KiB/s rd, 2.9 MiB/s wr, 129 op/s 2024-09-19T06:39:23.863 INFO:tasks.workunit.client.0.smithi053.stdout: 10 278535 26.98 MB/sec execute 427 sec latency 707.937 ms 2024-09-19T06:39:24.046 INFO:tasks.workunit.client.1.smithi155.stdout: 10 72774 7.08 MB/sec execute 430 sec latency 171.761 ms 2024-09-19T06:39:24.862 INFO:tasks.workunit.client.0.smithi053.stdout: 10 278955 26.95 MB/sec execute 428 sec latency 439.540 ms 2024-09-19T06:39:24.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:24 smithi155 ceph-mon[35292]: pgmap v897: 129 pgs: 129 active+clean; 419 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 9.2 KiB/s rd, 3.5 MiB/s wr, 187 op/s 2024-09-19T06:39:25.046 INFO:tasks.workunit.client.1.smithi155.stdout: 10 72906 7.07 MB/sec execute 431 sec latency 192.283 ms 2024-09-19T06:39:25.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:24 smithi053 ceph-mon[27968]: pgmap v897: 129 pgs: 129 active+clean; 419 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 9.2 KiB/s rd, 3.5 MiB/s wr, 187 op/s 2024-09-19T06:39:25.863 INFO:tasks.workunit.client.0.smithi053.stdout: 10 280015 27.07 MB/sec execute 429 sec latency 285.296 ms 2024-09-19T06:39:26.046 INFO:tasks.workunit.client.1.smithi155.stdout: 10 73108 7.06 MB/sec execute 432 sec latency 73.313 ms 2024-09-19T06:39:26.862 INFO:tasks.workunit.client.0.smithi053.stdout: 10 280995 27.12 MB/sec execute 430 sec latency 111.357 ms 2024-09-19T06:39:26.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:26 smithi155 ceph-mon[35292]: pgmap v898: 129 pgs: 129 active+clean; 419 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 6.8 KiB/s rd, 2.6 MiB/s wr, 150 op/s 2024-09-19T06:39:27.046 INFO:tasks.workunit.client.1.smithi155.stdout: 10 73335 7.07 MB/sec execute 433 sec latency 92.551 ms 2024-09-19T06:39:27.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:26 smithi053 ceph-mon[27968]: pgmap v898: 129 pgs: 129 active+clean; 419 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 6.8 KiB/s rd, 2.6 MiB/s wr, 150 op/s 2024-09-19T06:39:27.863 INFO:tasks.workunit.client.0.smithi053.stdout: 10 281219 27.07 MB/sec execute 431 sec latency 164.052 ms 2024-09-19T06:39:28.046 INFO:tasks.workunit.client.1.smithi155.stdout: 10 73511 7.07 MB/sec execute 434 sec latency 73.980 ms 2024-09-19T06:39:28.863 INFO:tasks.workunit.client.0.smithi053.stdout: 10 282277 27.13 MB/sec execute 432 sec latency 293.066 ms 2024-09-19T06:39:28.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:28 smithi155 ceph-mon[35292]: pgmap v899: 129 pgs: 129 active+clean; 422 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 6.8 KiB/s rd, 2.5 MiB/s wr, 169 op/s 2024-09-19T06:39:29.046 INFO:tasks.workunit.client.1.smithi155.stdout: 10 73629 7.07 MB/sec execute 435 sec latency 79.942 ms 2024-09-19T06:39:29.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:28 smithi053 ceph-mon[27968]: pgmap v899: 129 pgs: 129 active+clean; 422 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 6.8 KiB/s rd, 2.5 MiB/s wr, 169 op/s 2024-09-19T06:39:29.863 INFO:tasks.workunit.client.0.smithi053.stdout: 10 283308 27.23 MB/sec execute 433 sec latency 409.844 ms 2024-09-19T06:39:29.983 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:29 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:39:30.046 INFO:tasks.workunit.client.1.smithi155.stdout: 10 73777 7.09 MB/sec execute 436 sec latency 96.380 ms 2024-09-19T06:39:30.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:39:30.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:30 smithi053 ceph-mon[27968]: pgmap v900: 129 pgs: 129 active+clean; 424 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 8.3 KiB/s rd, 2.7 MiB/s wr, 194 op/s 2024-09-19T06:39:30.863 INFO:tasks.workunit.client.0.smithi053.stdout: 10 284480 27.31 MB/sec execute 434 sec latency 125.388 ms 2024-09-19T06:39:30.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:30 smithi155 ceph-mon[35292]: pgmap v900: 129 pgs: 129 active+clean; 424 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 8.3 KiB/s rd, 2.7 MiB/s wr, 194 op/s 2024-09-19T06:39:31.046 INFO:tasks.workunit.client.1.smithi155.stdout: 10 73997 7.13 MB/sec execute 437 sec latency 109.268 ms 2024-09-19T06:39:31.863 INFO:tasks.workunit.client.0.smithi053.stdout: 10 284538 27.25 MB/sec execute 435 sec latency 519.773 ms 2024-09-19T06:39:32.046 INFO:tasks.workunit.client.1.smithi155.stdout: 10 74238 7.15 MB/sec execute 438 sec latency 159.383 ms 2024-09-19T06:39:32.863 INFO:tasks.workunit.client.0.smithi053.stdout: 10 284591 27.19 MB/sec execute 436 sec latency 752.277 ms 2024-09-19T06:39:32.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:32 smithi155 ceph-mon[35292]: pgmap v901: 129 pgs: 129 active+clean; 426 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.8 KiB/s rd, 2.4 MiB/s wr, 143 op/s 2024-09-19T06:39:33.046 INFO:tasks.workunit.client.1.smithi155.stdout: 10 74392 7.15 MB/sec execute 439 sec latency 164.689 ms 2024-09-19T06:39:33.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:32 smithi053 ceph-mon[27968]: pgmap v901: 129 pgs: 129 active+clean; 426 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.8 KiB/s rd, 2.4 MiB/s wr, 143 op/s 2024-09-19T06:39:33.863 INFO:tasks.workunit.client.0.smithi053.stdout: 10 285443 27.24 MB/sec execute 437 sec latency 130.307 ms 2024-09-19T06:39:34.046 INFO:tasks.workunit.client.1.smithi155.stdout: 10 74440 7.14 MB/sec execute 440 sec latency 91.381 ms 2024-09-19T06:39:34.863 INFO:tasks.workunit.client.0.smithi053.stdout: 10 286668 27.33 MB/sec execute 438 sec latency 117.716 ms 2024-09-19T06:39:34.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:34 smithi155 ceph-mon[35292]: pgmap v902: 129 pgs: 129 active+clean; 442 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4.7 KiB/s rd, 4.2 MiB/s wr, 186 op/s 2024-09-19T06:39:35.046 INFO:tasks.workunit.client.1.smithi155.stdout: 10 74491 7.12 MB/sec execute 441 sec latency 138.606 ms 2024-09-19T06:39:35.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:34 smithi053 ceph-mon[27968]: pgmap v902: 129 pgs: 129 active+clean; 442 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4.7 KiB/s rd, 4.2 MiB/s wr, 186 op/s 2024-09-19T06:39:35.863 INFO:tasks.workunit.client.0.smithi053.stdout: 10 288014 27.43 MB/sec execute 439 sec latency 131.818 ms 2024-09-19T06:39:36.046 INFO:tasks.workunit.client.1.smithi155.stdout: 10 74627 7.14 MB/sec execute 442 sec latency 114.969 ms 2024-09-19T06:39:36.863 INFO:tasks.workunit.client.0.smithi053.stdout: 10 288137 27.38 MB/sec execute 440 sec latency 112.955 ms 2024-09-19T06:39:36.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:36 smithi155 ceph-mon[35292]: pgmap v903: 129 pgs: 129 active+clean; 445 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.3 KiB/s rd, 3.7 MiB/s wr, 131 op/s 2024-09-19T06:39:37.046 INFO:tasks.workunit.client.1.smithi155.stdout: 10 74813 7.16 MB/sec execute 443 sec latency 122.655 ms 2024-09-19T06:39:37.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:36 smithi053 ceph-mon[27968]: pgmap v903: 129 pgs: 129 active+clean; 445 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.3 KiB/s rd, 3.7 MiB/s wr, 131 op/s 2024-09-19T06:39:37.863 INFO:tasks.workunit.client.0.smithi053.stdout: 10 288263 27.32 MB/sec execute 441 sec latency 495.679 ms 2024-09-19T06:39:38.046 INFO:tasks.workunit.client.1.smithi155.stdout: 10 74923 7.15 MB/sec execute 444 sec latency 245.123 ms 2024-09-19T06:39:38.863 INFO:tasks.workunit.client.0.smithi053.stdout: 10 288496 27.29 MB/sec execute 442 sec latency 432.341 ms 2024-09-19T06:39:38.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:38 smithi155 ceph-mon[35292]: pgmap v904: 129 pgs: 129 active+clean; 445 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.3 KiB/s rd, 3.9 MiB/s wr, 126 op/s 2024-09-19T06:39:39.046 INFO:tasks.workunit.client.1.smithi155.stdout: 10 74999 7.15 MB/sec execute 445 sec latency 353.000 ms 2024-09-19T06:39:39.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:38 smithi053 ceph-mon[27968]: pgmap v904: 129 pgs: 129 active+clean; 445 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.3 KiB/s rd, 3.9 MiB/s wr, 126 op/s 2024-09-19T06:39:39.863 INFO:tasks.workunit.client.0.smithi053.stdout: 10 288991 27.30 MB/sec execute 443 sec latency 533.770 ms 2024-09-19T06:39:40.049 INFO:tasks.workunit.client.1.smithi155.stdout: 10 75071 7.13 MB/sec execute 446 sec latency 199.676 ms 2024-09-19T06:39:40.863 INFO:tasks.workunit.client.0.smithi053.stdout: 10 289174 27.24 MB/sec execute 444 sec latency 509.652 ms 2024-09-19T06:39:40.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:40 smithi155 ceph-mon[35292]: pgmap v905: 129 pgs: 129 active+clean; 449 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 5.8 KiB/s rd, 4.4 MiB/s wr, 147 op/s 2024-09-19T06:39:41.046 INFO:tasks.workunit.client.1.smithi155.stdout: 10 75139 7.12 MB/sec execute 447 sec latency 180.542 ms 2024-09-19T06:39:41.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:40 smithi053 ceph-mon[27968]: pgmap v905: 129 pgs: 129 active+clean; 449 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 5.8 KiB/s rd, 4.4 MiB/s wr, 147 op/s 2024-09-19T06:39:41.863 INFO:tasks.workunit.client.0.smithi053.stdout: 10 289598 27.21 MB/sec execute 445 sec latency 424.937 ms 2024-09-19T06:39:42.046 INFO:tasks.workunit.client.1.smithi155.stdout: 10 75225 7.11 MB/sec execute 448 sec latency 209.542 ms 2024-09-19T06:39:42.863 INFO:tasks.workunit.client.0.smithi053.stdout: 10 289817 27.18 MB/sec execute 446 sec latency 575.277 ms 2024-09-19T06:39:42.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:42 smithi155 ceph-mon[35292]: pgmap v906: 129 pgs: 129 active+clean; 448 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4.3 KiB/s rd, 4.3 MiB/s wr, 120 op/s 2024-09-19T06:39:43.046 INFO:tasks.workunit.client.1.smithi155.stdout: 10 75307 7.10 MB/sec execute 449 sec latency 210.485 ms 2024-09-19T06:39:43.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:42 smithi053 ceph-mon[27968]: pgmap v906: 129 pgs: 129 active+clean; 448 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4.3 KiB/s rd, 4.3 MiB/s wr, 120 op/s 2024-09-19T06:39:43.863 INFO:tasks.workunit.client.0.smithi053.stdout: 10 290099 27.18 MB/sec execute 447 sec latency 373.741 ms 2024-09-19T06:39:44.047 INFO:tasks.workunit.client.1.smithi155.stdout: 10 75408 7.09 MB/sec execute 450 sec latency 208.385 ms 2024-09-19T06:39:44.863 INFO:tasks.workunit.client.0.smithi053.stdout: 10 290405 27.17 MB/sec execute 448 sec latency 504.650 ms 2024-09-19T06:39:45.047 INFO:tasks.workunit.client.1.smithi155.stdout: 10 75509 7.08 MB/sec execute 451 sec latency 189.091 ms 2024-09-19T06:39:45.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:44 smithi053 ceph-mon[27968]: pgmap v907: 129 pgs: 129 active+clean; 450 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 7.5 KiB/s rd, 4.7 MiB/s wr, 144 op/s 2024-09-19T06:39:45.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:44 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:39:45.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:44 smithi155 ceph-mon[35292]: pgmap v907: 129 pgs: 129 active+clean; 450 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 7.5 KiB/s rd, 4.7 MiB/s wr, 144 op/s 2024-09-19T06:39:45.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:44 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:39:45.864 INFO:tasks.workunit.client.0.smithi053.stdout: 10 290614 27.13 MB/sec execute 449 sec latency 395.549 ms 2024-09-19T06:39:46.047 INFO:tasks.workunit.client.1.smithi155.stdout: 10 75613 7.07 MB/sec execute 452 sec latency 233.246 ms 2024-09-19T06:39:46.863 INFO:tasks.workunit.client.0.smithi053.stdout: 10 290834 27.10 MB/sec execute 450 sec latency 490.735 ms 2024-09-19T06:39:47.047 INFO:tasks.workunit.client.1.smithi155.stdout: 10 75725 7.06 MB/sec execute 453 sec latency 167.795 ms 2024-09-19T06:39:47.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:46 smithi053 ceph-mon[27968]: pgmap v908: 129 pgs: 129 active+clean; 448 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 5.7 KiB/s rd, 2.9 MiB/s wr, 97 op/s 2024-09-19T06:39:47.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:46 smithi155 ceph-mon[35292]: pgmap v908: 129 pgs: 129 active+clean; 448 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 5.7 KiB/s rd, 2.9 MiB/s wr, 97 op/s 2024-09-19T06:39:47.864 INFO:tasks.workunit.client.0.smithi053.stdout: 10 291554 27.13 MB/sec execute 451 sec latency 418.446 ms 2024-09-19T06:39:48.047 INFO:tasks.workunit.client.1.smithi155.stdout: 10 75829 7.06 MB/sec execute 454 sec latency 206.176 ms 2024-09-19T06:39:48.864 INFO:tasks.workunit.client.0.smithi053.stdout: 10 291559 27.07 MB/sec execute 452 sec latency 417.067 ms 2024-09-19T06:39:49.047 INFO:tasks.workunit.client.1.smithi155.stdout: 10 75958 7.07 MB/sec execute 455 sec latency 165.840 ms 2024-09-19T06:39:49.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:48 smithi053 ceph-mon[27968]: pgmap v909: 129 pgs: 129 active+clean; 443 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 5.7 KiB/s rd, 2.6 MiB/s wr, 105 op/s 2024-09-19T06:39:49.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:48 smithi155 ceph-mon[35292]: pgmap v909: 129 pgs: 129 active+clean; 443 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 5.7 KiB/s rd, 2.6 MiB/s wr, 105 op/s 2024-09-19T06:39:49.864 INFO:tasks.workunit.client.0.smithi053.stdout: 10 291613 27.01 MB/sec execute 453 sec latency 492.531 ms 2024-09-19T06:39:50.047 INFO:tasks.workunit.client.1.smithi155.stdout: 10 76105 7.07 MB/sec execute 456 sec latency 150.587 ms 2024-09-19T06:39:50.864 INFO:tasks.workunit.client.0.smithi053.stdout: 10 291952 26.98 MB/sec execute 454 sec latency 100.476 ms 2024-09-19T06:39:51.047 INFO:tasks.workunit.client.1.smithi155.stdout: 10 76264 7.08 MB/sec execute 457 sec latency 83.155 ms 2024-09-19T06:39:51.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:50 smithi053 ceph-mon[27968]: pgmap v910: 129 pgs: 129 active+clean; 435 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 9.2 KiB/s rd, 2.7 MiB/s wr, 160 op/s 2024-09-19T06:39:51.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:50 smithi155 ceph-mon[35292]: pgmap v910: 129 pgs: 129 active+clean; 435 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 9.2 KiB/s rd, 2.7 MiB/s wr, 160 op/s 2024-09-19T06:39:51.864 INFO:tasks.workunit.client.0.smithi053.stdout: 10 293270 27.04 MB/sec execute 455 sec latency 123.485 ms 2024-09-19T06:39:52.047 INFO:tasks.workunit.client.1.smithi155.stdout: 10 76434 7.07 MB/sec execute 458 sec latency 101.109 ms 2024-09-19T06:39:52.864 INFO:tasks.workunit.client.0.smithi053.stdout: 10 294367 27.16 MB/sec execute 456 sec latency 134.923 ms 2024-09-19T06:39:53.047 INFO:tasks.workunit.client.1.smithi155.stdout: 10 76642 7.07 MB/sec execute 459 sec latency 103.516 ms 2024-09-19T06:39:53.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:52 smithi053 ceph-mon[27968]: pgmap v911: 129 pgs: 129 active+clean; 433 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 6.7 KiB/s rd, 2.0 MiB/s wr, 120 op/s 2024-09-19T06:39:53.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:52 smithi155 ceph-mon[35292]: pgmap v911: 129 pgs: 129 active+clean; 433 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 6.7 KiB/s rd, 2.0 MiB/s wr, 120 op/s 2024-09-19T06:39:53.864 INFO:tasks.workunit.client.0.smithi053.stdout: 10 295163 27.19 MB/sec execute 457 sec latency 226.729 ms 2024-09-19T06:39:54.047 INFO:tasks.workunit.client.1.smithi155.stdout: 10 76847 7.07 MB/sec execute 460 sec latency 86.832 ms 2024-09-19T06:39:54.864 INFO:tasks.workunit.client.0.smithi053.stdout: 10 295339 27.14 MB/sec execute 458 sec latency 197.071 ms 2024-09-19T06:39:55.047 INFO:tasks.workunit.client.1.smithi155.stdout: 10 77013 7.07 MB/sec execute 461 sec latency 133.074 ms 2024-09-19T06:39:55.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:54 smithi053 ceph-mon[27968]: pgmap v912: 129 pgs: 129 active+clean; 435 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 8.3 KiB/s rd, 2.5 MiB/s wr, 170 op/s 2024-09-19T06:39:55.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:54 smithi155 ceph-mon[35292]: pgmap v912: 129 pgs: 129 active+clean; 435 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 8.3 KiB/s rd, 2.5 MiB/s wr, 170 op/s 2024-09-19T06:39:55.864 INFO:tasks.workunit.client.0.smithi053.stdout: 10 295482 27.10 MB/sec execute 459 sec latency 492.603 ms 2024-09-19T06:39:56.047 INFO:tasks.workunit.client.1.smithi155.stdout: 10 77084 7.06 MB/sec execute 462 sec latency 183.651 ms 2024-09-19T06:39:56.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:55 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:39:56.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:55 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:39:56.864 INFO:tasks.workunit.client.0.smithi053.stdout: 10 296007 27.12 MB/sec execute 460 sec latency 536.877 ms 2024-09-19T06:39:57.047 INFO:tasks.workunit.client.1.smithi155.stdout: 10 77172 7.06 MB/sec execute 463 sec latency 217.666 ms 2024-09-19T06:39:57.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:56 smithi053 ceph-mon[27968]: pgmap v913: 129 pgs: 129 active+clean; 437 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.0 MiB/s wr, 154 op/s 2024-09-19T06:39:57.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:56 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:57.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:56 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:39:57.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:56 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:39:57.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:56 smithi155 ceph-mon[35292]: pgmap v913: 129 pgs: 129 active+clean; 437 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.0 MiB/s wr, 154 op/s 2024-09-19T06:39:57.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:56 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:57.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:56 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:39:57.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:56 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:39:57.864 INFO:tasks.workunit.client.0.smithi053.stdout: 10 296140 27.06 MB/sec execute 461 sec latency 599.128 ms 2024-09-19T06:39:58.047 INFO:tasks.workunit.client.1.smithi155.stdout: 10 77240 7.05 MB/sec execute 464 sec latency 247.347 ms 2024-09-19T06:39:58.864 INFO:tasks.workunit.client.0.smithi053.stdout: 10 296462 27.02 MB/sec execute 462 sec latency 455.018 ms 2024-09-19T06:39:59.047 INFO:tasks.workunit.client.1.smithi155.stdout: 10 77396 7.07 MB/sec execute 465 sec latency 178.237 ms 2024-09-19T06:39:59.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:58 smithi053 ceph-mon[27968]: pgmap v914: 129 pgs: 129 active+clean; 440 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.4 MiB/s wr, 163 op/s 2024-09-19T06:39:59.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:58 smithi155 ceph-mon[35292]: pgmap v914: 129 pgs: 129 active+clean; 440 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.4 MiB/s wr, 163 op/s 2024-09-19T06:39:59.864 INFO:tasks.workunit.client.0.smithi053.stdout: 10 296816 26.99 MB/sec execute 463 sec latency 552.231 ms 2024-09-19T06:40:00.047 INFO:tasks.workunit.client.1.smithi155.stdout: 10 77571 7.09 MB/sec execute 466 sec latency 189.762 ms 2024-09-19T06:40:00.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:39:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:40:00.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:39:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:40:00.864 INFO:tasks.workunit.client.0.smithi053.stdout: 10 297014 26.96 MB/sec execute 464 sec latency 501.557 ms 2024-09-19T06:40:01.047 INFO:tasks.workunit.client.1.smithi155.stdout: 10 77754 7.11 MB/sec execute 467 sec latency 182.561 ms 2024-09-19T06:40:01.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:00 smithi053 ceph-mon[27968]: pgmap v915: 129 pgs: 129 active+clean; 445 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 6.6 KiB/s rd, 3.1 MiB/s wr, 184 op/s 2024-09-19T06:40:01.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:00 smithi053 ceph-mon[27968]: overall HEALTH_OK 2024-09-19T06:40:01.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:00 smithi155 ceph-mon[35292]: pgmap v915: 129 pgs: 129 active+clean; 445 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 6.6 KiB/s rd, 3.1 MiB/s wr, 184 op/s 2024-09-19T06:40:01.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:00 smithi155 ceph-mon[35292]: overall HEALTH_OK 2024-09-19T06:40:01.864 INFO:tasks.workunit.client.0.smithi053.stdout: 10 297923 27.05 MB/sec execute 465 sec latency 417.258 ms 2024-09-19T06:40:02.047 INFO:tasks.workunit.client.1.smithi155.stdout: 10 77935 7.12 MB/sec execute 468 sec latency 147.294 ms 2024-09-19T06:40:02.864 INFO:tasks.workunit.client.0.smithi053.stdout: 10 298725 27.08 MB/sec execute 466 sec latency 109.874 ms 2024-09-19T06:40:03.047 INFO:tasks.workunit.client.1.smithi155.stdout: 10 77987 7.10 MB/sec execute 469 sec latency 75.742 ms 2024-09-19T06:40:03.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:02 smithi053 ceph-mon[27968]: pgmap v916: 129 pgs: 129 active+clean; 446 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.7 MiB/s wr, 127 op/s 2024-09-19T06:40:03.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:02 smithi155 ceph-mon[35292]: pgmap v916: 129 pgs: 129 active+clean; 446 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.7 MiB/s wr, 127 op/s 2024-09-19T06:40:03.864 INFO:tasks.workunit.client.0.smithi053.stdout: 10 298882 27.03 MB/sec execute 467 sec latency 362.514 ms 2024-09-19T06:40:04.047 INFO:tasks.workunit.client.1.smithi155.stdout: 10 78044 7.09 MB/sec execute 470 sec latency 111.988 ms 2024-09-19T06:40:04.864 INFO:tasks.workunit.client.0.smithi053.stdout: 10 299829 27.08 MB/sec execute 468 sec latency 262.086 ms 2024-09-19T06:40:05.047 INFO:tasks.workunit.client.1.smithi155.stdout: 10 78198 7.10 MB/sec execute 471 sec latency 89.474 ms 2024-09-19T06:40:05.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:04 smithi053 ceph-mon[27968]: pgmap v917: 129 pgs: 129 active+clean; 459 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 4.8 KiB/s rd, 3.9 MiB/s wr, 157 op/s 2024-09-19T06:40:05.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:04 smithi155 ceph-mon[35292]: pgmap v917: 129 pgs: 129 active+clean; 459 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 4.8 KiB/s rd, 3.9 MiB/s wr, 157 op/s 2024-09-19T06:40:05.881 INFO:tasks.workunit.client.0.smithi053.stdout: 10 301022 27.17 MB/sec execute 469 sec latency 184.520 ms 2024-09-19T06:40:06.048 INFO:tasks.workunit.client.1.smithi155.stdout: 10 78375 7.12 MB/sec execute 472 sec latency 102.766 ms 2024-09-19T06:40:06.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:06 smithi053 ceph-mon[27968]: pgmap v918: 129 pgs: 129 active+clean; 462 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 3.5 MiB/s wr, 107 op/s 2024-09-19T06:40:06.864 INFO:tasks.workunit.client.0.smithi053.stdout: 10 301756 27.20 MB/sec execute 470 sec latency 359.557 ms 2024-09-19T06:40:07.047 INFO:tasks.workunit.client.1.smithi155.stdout: 10 78499 7.12 MB/sec execute 473 sec latency 149.934 ms 2024-09-19T06:40:07.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:06 smithi155 ceph-mon[35292]: pgmap v918: 129 pgs: 129 active+clean; 462 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 3.5 MiB/s wr, 107 op/s 2024-09-19T06:40:07.864 INFO:tasks.workunit.client.0.smithi053.stdout: 10 302208 27.19 MB/sec execute 471 sec latency 532.538 ms 2024-09-19T06:40:08.047 INFO:tasks.workunit.client.1.smithi155.stdout: 10 78586 7.12 MB/sec execute 474 sec latency 154.713 ms 2024-09-19T06:40:08.864 INFO:tasks.workunit.client.0.smithi053.stdout: 10 302218 27.14 MB/sec execute 472 sec latency 693.273 ms 2024-09-19T06:40:09.047 INFO:tasks.workunit.client.1.smithi155.stdout: 10 78654 7.10 MB/sec execute 475 sec latency 214.694 ms 2024-09-19T06:40:09.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:08 smithi053 ceph-mon[27968]: pgmap v919: 129 pgs: 129 active+clean; 460 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 3.8 MiB/s wr, 103 op/s 2024-09-19T06:40:09.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:08 smithi155 ceph-mon[35292]: pgmap v919: 129 pgs: 129 active+clean; 460 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 3.8 MiB/s wr, 103 op/s 2024-09-19T06:40:09.864 INFO:tasks.workunit.client.0.smithi053.stdout: 10 302232 27.08 MB/sec execute 473 sec latency 763.326 ms 2024-09-19T06:40:10.048 INFO:tasks.workunit.client.1.smithi155.stdout: 10 78758 7.09 MB/sec execute 476 sec latency 139.324 ms 2024-09-19T06:40:10.865 INFO:tasks.workunit.client.0.smithi053.stdout: 10 302261 27.03 MB/sec execute 474 sec latency 426.608 ms 2024-09-19T06:40:11.034 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:10 smithi155 ceph-mon[35292]: pgmap v920: 129 pgs: 129 active+clean; 442 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 6.1 KiB/s rd, 4.2 MiB/s wr, 135 op/s 2024-09-19T06:40:11.047 INFO:tasks.workunit.client.1.smithi155.stdout: 10 78851 7.09 MB/sec execute 477 sec latency 174.453 ms 2024-09-19T06:40:11.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:10 smithi053 ceph-mon[27968]: pgmap v920: 129 pgs: 129 active+clean; 442 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 6.1 KiB/s rd, 4.2 MiB/s wr, 135 op/s 2024-09-19T06:40:11.865 INFO:tasks.workunit.client.0.smithi053.stdout: 10 302283 26.97 MB/sec execute 475 sec latency 367.515 ms 2024-09-19T06:40:12.048 INFO:tasks.workunit.client.1.smithi155.stdout: 10 78951 7.08 MB/sec execute 478 sec latency 238.155 ms 2024-09-19T06:40:12.865 INFO:tasks.workunit.client.0.smithi053.stdout: 10 302326 26.92 MB/sec execute 476 sec latency 392.387 ms 2024-09-19T06:40:13.048 INFO:tasks.workunit.client.1.smithi155.stdout: 10 79075 7.07 MB/sec execute 479 sec latency 190.646 ms 2024-09-19T06:40:13.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:12 smithi053 ceph-mon[27968]: pgmap v921: 129 pgs: 129 active+clean; 439 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 4.7 KiB/s rd, 3.9 MiB/s wr, 106 op/s 2024-09-19T06:40:13.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:12 smithi155 ceph-mon[35292]: pgmap v921: 129 pgs: 129 active+clean; 439 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 4.7 KiB/s rd, 3.9 MiB/s wr, 106 op/s 2024-09-19T06:40:13.865 INFO:tasks.workunit.client.0.smithi053.stdout: 10 302346 26.86 MB/sec execute 477 sec latency 467.332 ms 2024-09-19T06:40:14.048 INFO:tasks.workunit.client.1.smithi155.stdout: 10 79193 7.06 MB/sec execute 480 sec latency 184.009 ms 2024-09-19T06:40:14.865 INFO:tasks.workunit.client.0.smithi053.stdout: 10 303136 26.90 MB/sec execute 478 sec latency 374.717 ms 2024-09-19T06:40:15.048 INFO:tasks.workunit.client.1.smithi155.stdout: 10 79342 7.06 MB/sec execute 481 sec latency 163.302 ms 2024-09-19T06:40:15.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:14 smithi053 ceph-mon[27968]: pgmap v922: 129 pgs: 129 active+clean; 439 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 8.7 KiB/s rd, 4.2 MiB/s wr, 150 op/s 2024-09-19T06:40:15.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:14 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:40:15.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:14 smithi155 ceph-mon[35292]: pgmap v922: 129 pgs: 129 active+clean; 439 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 8.7 KiB/s rd, 4.2 MiB/s wr, 150 op/s 2024-09-19T06:40:15.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:14 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:40:15.865 INFO:tasks.workunit.client.0.smithi053.stdout: 10 304205 26.94 MB/sec execute 479 sec latency 135.919 ms 2024-09-19T06:40:16.048 INFO:tasks.workunit.client.1.smithi155.stdout: 10 79519 7.07 MB/sec execute 482 sec latency 87.826 ms 2024-09-19T06:40:16.865 INFO:tasks.workunit.client.0.smithi053.stdout: 10 305745 27.08 MB/sec execute 480 sec latency 154.709 ms 2024-09-19T06:40:17.048 INFO:tasks.workunit.client.1.smithi155.stdout: 10 79680 7.08 MB/sec execute 483 sec latency 107.940 ms 2024-09-19T06:40:17.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:16 smithi053 ceph-mon[27968]: pgmap v923: 129 pgs: 129 active+clean; 437 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 6.8 KiB/s rd, 2.9 MiB/s wr, 122 op/s 2024-09-19T06:40:17.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:16 smithi155 ceph-mon[35292]: pgmap v923: 129 pgs: 129 active+clean; 437 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 6.8 KiB/s rd, 2.9 MiB/s wr, 122 op/s 2024-09-19T06:40:17.865 INFO:tasks.workunit.client.0.smithi053.stdout: 10 305869 27.03 MB/sec execute 481 sec latency 108.307 ms 2024-09-19T06:40:18.048 INFO:tasks.workunit.client.1.smithi155.stdout: 10 79834 7.08 MB/sec execute 484 sec latency 82.018 ms 2024-09-19T06:40:18.865 INFO:tasks.workunit.client.0.smithi053.stdout: 10 306671 27.07 MB/sec execute 482 sec latency 116.567 ms 2024-09-19T06:40:19.048 INFO:tasks.workunit.client.1.smithi155.stdout: 10 80004 7.07 MB/sec execute 485 sec latency 90.696 ms 2024-09-19T06:40:19.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:18 smithi053 ceph-mon[27968]: pgmap v924: 129 pgs: 129 active+clean; 431 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 6.8 KiB/s rd, 2.9 MiB/s wr, 135 op/s 2024-09-19T06:40:19.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:18 smithi155 ceph-mon[35292]: pgmap v924: 129 pgs: 129 active+clean; 431 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 6.8 KiB/s rd, 2.9 MiB/s wr, 135 op/s 2024-09-19T06:40:19.865 INFO:tasks.workunit.client.0.smithi053.stdout: 10 307116 27.03 MB/sec execute 483 sec latency 370.100 ms 2024-09-19T06:40:20.048 INFO:tasks.workunit.client.1.smithi155.stdout: 10 80165 7.07 MB/sec execute 486 sec latency 187.859 ms 2024-09-19T06:40:20.865 INFO:tasks.workunit.client.0.smithi053.stdout: 10 307556 27.02 MB/sec execute 484 sec latency 525.716 ms 2024-09-19T06:40:21.048 INFO:tasks.workunit.client.1.smithi155.stdout: 10 80312 7.06 MB/sec execute 487 sec latency 178.577 ms 2024-09-19T06:40:21.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:20 smithi053 ceph-mon[27968]: pgmap v925: 129 pgs: 129 active+clean; 427 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 9.5 KiB/s rd, 3.2 MiB/s wr, 187 op/s 2024-09-19T06:40:21.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:20 smithi155 ceph-mon[35292]: pgmap v925: 129 pgs: 129 active+clean; 427 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 9.5 KiB/s rd, 3.2 MiB/s wr, 187 op/s 2024-09-19T06:40:21.865 INFO:tasks.workunit.client.0.smithi053.stdout: 10 307893 27.04 MB/sec execute 485 sec latency 421.388 ms 2024-09-19T06:40:22.048 INFO:tasks.workunit.client.1.smithi155.stdout: 10 80432 7.06 MB/sec execute 488 sec latency 184.472 ms 2024-09-19T06:40:22.865 INFO:tasks.workunit.client.0.smithi053.stdout: 10 308110 27.01 MB/sec execute 486 sec latency 535.305 ms 2024-09-19T06:40:23.048 INFO:tasks.workunit.client.1.smithi155.stdout: 10 80547 7.05 MB/sec execute 489 sec latency 200.010 ms 2024-09-19T06:40:23.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:22 smithi053 ceph-mon[27968]: pgmap v926: 129 pgs: 129 active+clean; 426 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 6.7 KiB/s rd, 2.4 MiB/s wr, 147 op/s 2024-09-19T06:40:23.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:22 smithi155 ceph-mon[35292]: pgmap v926: 129 pgs: 129 active+clean; 426 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 6.7 KiB/s rd, 2.4 MiB/s wr, 147 op/s 2024-09-19T06:40:23.868 INFO:tasks.workunit.client.0.smithi053.stdout: 10 308357 26.98 MB/sec execute 487 sec latency 476.775 ms 2024-09-19T06:40:24.048 INFO:tasks.workunit.client.1.smithi155.stdout: 10 80653 7.05 MB/sec execute 490 sec latency 165.726 ms 2024-09-19T06:40:24.865 INFO:tasks.workunit.client.0.smithi053.stdout: 10 308578 26.95 MB/sec execute 488 sec latency 469.081 ms 2024-09-19T06:40:25.048 INFO:tasks.workunit.client.1.smithi155.stdout: 10 80721 7.04 MB/sec execute 491 sec latency 264.153 ms 2024-09-19T06:40:25.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:24 smithi053 ceph-mon[27968]: pgmap v927: 129 pgs: 129 active+clean; 431 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 8.3 KiB/s rd, 2.7 MiB/s wr, 191 op/s 2024-09-19T06:40:25.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:24 smithi155 ceph-mon[35292]: pgmap v927: 129 pgs: 129 active+clean; 431 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 8.3 KiB/s rd, 2.7 MiB/s wr, 191 op/s 2024-09-19T06:40:25.865 INFO:tasks.workunit.client.0.smithi053.stdout: 10 309364 26.98 MB/sec execute 489 sec latency 234.006 ms 2024-09-19T06:40:26.048 INFO:tasks.workunit.client.1.smithi155.stdout: 10 80876 7.06 MB/sec execute 492 sec latency 152.325 ms 2024-09-19T06:40:26.866 INFO:tasks.workunit.client.0.smithi053.stdout: 10 309732 26.95 MB/sec execute 490 sec latency 93.236 ms 2024-09-19T06:40:27.048 INFO:tasks.workunit.client.1.smithi155.stdout: 10 81087 7.09 MB/sec execute 493 sec latency 108.357 ms 2024-09-19T06:40:27.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:26 smithi053 ceph-mon[27968]: pgmap v928: 129 pgs: 129 active+clean; 432 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 4.3 KiB/s rd, 2.3 MiB/s wr, 152 op/s 2024-09-19T06:40:27.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:26 smithi155 ceph-mon[35292]: pgmap v928: 129 pgs: 129 active+clean; 432 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 4.3 KiB/s rd, 2.3 MiB/s wr, 152 op/s 2024-09-19T06:40:27.865 INFO:tasks.workunit.client.0.smithi053.stdout: 10 311149 27.03 MB/sec execute 491 sec latency 111.281 ms 2024-09-19T06:40:28.048 INFO:tasks.workunit.client.1.smithi155.stdout: 10 81333 7.11 MB/sec execute 494 sec latency 113.164 ms 2024-09-19T06:40:28.865 INFO:tasks.workunit.client.0.smithi053.stdout: 10 312125 27.12 MB/sec execute 492 sec latency 136.347 ms 2024-09-19T06:40:29.048 INFO:tasks.workunit.client.1.smithi155.stdout: 10 81500 7.12 MB/sec execute 495 sec latency 88.551 ms 2024-09-19T06:40:29.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:28 smithi053 ceph-mon[27968]: pgmap v929: 129 pgs: 129 active+clean; 436 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 4.3 KiB/s rd, 2.8 MiB/s wr, 153 op/s 2024-09-19T06:40:29.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:28 smithi155 ceph-mon[35292]: pgmap v929: 129 pgs: 129 active+clean; 436 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 4.3 KiB/s rd, 2.8 MiB/s wr, 153 op/s 2024-09-19T06:40:29.866 INFO:tasks.workunit.client.0.smithi053.stdout: 10 312892 27.15 MB/sec execute 493 sec latency 346.736 ms 2024-09-19T06:40:30.048 INFO:tasks.workunit.client.1.smithi155.stdout: 10 81540 7.10 MB/sec execute 496 sec latency 140.098 ms 2024-09-19T06:40:30.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:40:30.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:29 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:40:30.865 INFO:tasks.workunit.client.0.smithi053.stdout: 10 312909 27.09 MB/sec execute 494 sec latency 390.587 ms 2024-09-19T06:40:31.048 INFO:tasks.workunit.client.1.smithi155.stdout: 10 81587 7.09 MB/sec execute 497 sec latency 188.946 ms 2024-09-19T06:40:31.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:30 smithi053 ceph-mon[27968]: pgmap v930: 129 pgs: 129 active+clean; 447 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.8 KiB/s rd, 3.8 MiB/s wr, 173 op/s 2024-09-19T06:40:31.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:30 smithi155 ceph-mon[35292]: pgmap v930: 129 pgs: 129 active+clean; 447 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.8 KiB/s rd, 3.8 MiB/s wr, 173 op/s 2024-09-19T06:40:31.869 INFO:tasks.workunit.client.0.smithi053.stdout: 10 312957 27.04 MB/sec execute 495 sec latency 425.581 ms 2024-09-19T06:40:32.048 INFO:tasks.workunit.client.1.smithi155.stdout: 10 81669 7.09 MB/sec execute 498 sec latency 192.525 ms 2024-09-19T06:40:32.865 INFO:tasks.workunit.client.0.smithi053.stdout: 10 312978 26.99 MB/sec execute 496 sec latency 409.926 ms 2024-09-19T06:40:33.048 INFO:tasks.workunit.client.1.smithi155.stdout: 10 81757 7.09 MB/sec execute 499 sec latency 174.269 ms 2024-09-19T06:40:33.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:32 smithi053 ceph-mon[27968]: pgmap v931: 129 pgs: 129 active+clean; 448 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.2 KiB/s rd, 3.3 MiB/s wr, 115 op/s 2024-09-19T06:40:33.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:32 smithi155 ceph-mon[35292]: pgmap v931: 129 pgs: 129 active+clean; 448 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.2 KiB/s rd, 3.3 MiB/s wr, 115 op/s 2024-09-19T06:40:33.865 INFO:tasks.workunit.client.0.smithi053.stdout: 10 312997 26.93 MB/sec execute 497 sec latency 408.891 ms 2024-09-19T06:40:34.048 INFO:tasks.workunit.client.1.smithi155.stdout: 10 81875 7.10 MB/sec execute 500 sec latency 194.451 ms 2024-09-19T06:40:34.867 INFO:tasks.workunit.client.0.smithi053.stdout: 10 314111 26.98 MB/sec execute 498 sec latency 384.264 ms 2024-09-19T06:40:35.048 INFO:tasks.workunit.client.1.smithi155.stdout: 10 82033 7.11 MB/sec execute 501 sec latency 173.928 ms 2024-09-19T06:40:35.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:34 smithi053 ceph-mon[27968]: pgmap v932: 129 pgs: 129 active+clean; 453 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.6 KiB/s rd, 3.9 MiB/s wr, 143 op/s 2024-09-19T06:40:35.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:34 smithi155 ceph-mon[35292]: pgmap v932: 129 pgs: 129 active+clean; 453 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.6 KiB/s rd, 3.9 MiB/s wr, 143 op/s 2024-09-19T06:40:35.866 INFO:tasks.workunit.client.0.smithi053.stdout: 10 315114 27.07 MB/sec execute 499 sec latency 203.168 ms 2024-09-19T06:40:36.048 INFO:tasks.workunit.client.1.smithi155.stdout: 10 82148 7.10 MB/sec execute 502 sec latency 101.395 ms 2024-09-19T06:40:36.866 INFO:tasks.workunit.client.0.smithi053.stdout: 10 316384 27.15 MB/sec execute 500 sec latency 190.637 ms 2024-09-19T06:40:37.049 INFO:tasks.workunit.client.1.smithi155.stdout: 10 82255 7.09 MB/sec execute 503 sec latency 92.543 ms 2024-09-19T06:40:37.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:36 smithi053 ceph-mon[27968]: pgmap v933: 129 pgs: 129 active+clean; 453 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.9 KiB/s rd, 3.6 MiB/s wr, 100 op/s 2024-09-19T06:40:37.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:36 smithi155 ceph-mon[35292]: pgmap v933: 129 pgs: 129 active+clean; 453 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.9 KiB/s rd, 3.6 MiB/s wr, 100 op/s 2024-09-19T06:40:37.866 INFO:tasks.workunit.client.0.smithi053.stdout: 10 316509 27.10 MB/sec execute 501 sec latency 92.031 ms 2024-09-19T06:40:38.049 INFO:tasks.workunit.client.1.smithi155.stdout: 10 82365 7.09 MB/sec execute 504 sec latency 167.591 ms 2024-09-19T06:40:38.866 INFO:tasks.workunit.client.0.smithi053.stdout: 10 317378 27.14 MB/sec execute 502 sec latency 103.212 ms 2024-09-19T06:40:39.049 INFO:tasks.workunit.client.1.smithi155.stdout: 10 82505 7.08 MB/sec execute 505 sec latency 123.140 ms 2024-09-19T06:40:39.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:38 smithi053 ceph-mon[27968]: pgmap v934: 129 pgs: 129 active+clean; 453 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.9 KiB/s rd, 3.9 MiB/s wr, 98 op/s 2024-09-19T06:40:39.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:38 smithi155 ceph-mon[35292]: pgmap v934: 129 pgs: 129 active+clean; 453 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.9 KiB/s rd, 3.9 MiB/s wr, 98 op/s 2024-09-19T06:40:39.866 INFO:tasks.workunit.client.0.smithi053.stdout: 10 318109 27.13 MB/sec execute 503 sec latency 416.285 ms 2024-09-19T06:40:40.049 INFO:tasks.workunit.client.1.smithi155.stdout: 10 82616 7.07 MB/sec execute 506 sec latency 161.474 ms 2024-09-19T06:40:40.868 INFO:tasks.workunit.client.0.smithi053.stdout: 10 318228 27.09 MB/sec execute 504 sec latency 531.352 ms 2024-09-19T06:40:41.049 INFO:tasks.workunit.client.1.smithi155.stdout: 10 82720 7.06 MB/sec execute 507 sec latency 220.011 ms 2024-09-19T06:40:41.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:40 smithi053 ceph-mon[27968]: pgmap v935: 129 pgs: 129 active+clean; 456 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.1 KiB/s rd, 4.1 MiB/s wr, 128 op/s 2024-09-19T06:40:41.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:40 smithi155 ceph-mon[35292]: pgmap v935: 129 pgs: 129 active+clean; 456 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.1 KiB/s rd, 4.1 MiB/s wr, 128 op/s 2024-09-19T06:40:41.867 INFO:tasks.workunit.client.0.smithi053.stdout: 10 318662 27.12 MB/sec execute 505 sec latency 607.508 ms 2024-09-19T06:40:42.050 INFO:tasks.workunit.client.1.smithi155.stdout: 10 82828 7.05 MB/sec execute 508 sec latency 166.754 ms 2024-09-19T06:40:42.866 INFO:tasks.workunit.client.0.smithi053.stdout: 10 318760 27.08 MB/sec execute 506 sec latency 563.755 ms 2024-09-19T06:40:43.049 INFO:tasks.workunit.client.1.smithi155.stdout: 10 82944 7.05 MB/sec execute 509 sec latency 144.116 ms 2024-09-19T06:40:43.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:42 smithi053 ceph-mon[27968]: pgmap v936: 129 pgs: 129 active+clean; 457 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.6 KiB/s rd, 3.0 MiB/s wr, 93 op/s 2024-09-19T06:40:43.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:42 smithi155 ceph-mon[35292]: pgmap v936: 129 pgs: 129 active+clean; 457 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.6 KiB/s rd, 3.0 MiB/s wr, 93 op/s 2024-09-19T06:40:43.866 INFO:tasks.workunit.client.0.smithi053.stdout: 10 318996 27.05 MB/sec execute 507 sec latency 431.940 ms 2024-09-19T06:40:44.049 INFO:tasks.workunit.client.1.smithi155.stdout: 10 83061 7.06 MB/sec execute 510 sec latency 168.991 ms 2024-09-19T06:40:44.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:43 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:40:44.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:43 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:40:44.866 INFO:tasks.workunit.client.0.smithi053.stdout: 10 319215 27.02 MB/sec execute 508 sec latency 512.277 ms 2024-09-19T06:40:45.049 INFO:tasks.workunit.client.1.smithi155.stdout: 10 83196 7.05 MB/sec execute 511 sec latency 166.096 ms 2024-09-19T06:40:45.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:44 smithi053 ceph-mon[27968]: pgmap v937: 129 pgs: 129 active+clean; 445 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 9.7 KiB/s rd, 3.4 MiB/s wr, 165 op/s 2024-09-19T06:40:45.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:44 smithi155 ceph-mon[35292]: pgmap v937: 129 pgs: 129 active+clean; 445 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 9.7 KiB/s rd, 3.4 MiB/s wr, 165 op/s 2024-09-19T06:40:45.866 INFO:tasks.workunit.client.0.smithi053.stdout: 10 319992 27.05 MB/sec execute 509 sec latency 466.448 ms 2024-09-19T06:40:46.049 INFO:tasks.workunit.client.1.smithi155.stdout: 10 83359 7.06 MB/sec execute 512 sec latency 102.058 ms 2024-09-19T06:40:46.866 INFO:tasks.workunit.client.0.smithi053.stdout: 10 320042 27.00 MB/sec execute 510 sec latency 549.236 ms 2024-09-19T06:40:47.049 INFO:tasks.workunit.client.1.smithi155.stdout: 10 83532 7.06 MB/sec execute 513 sec latency 93.243 ms 2024-09-19T06:40:47.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:46 smithi053 ceph-mon[27968]: pgmap v938: 129 pgs: 129 active+clean; 440 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.2 KiB/s rd, 2.8 MiB/s wr, 143 op/s 2024-09-19T06:40:47.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:46 smithi155 ceph-mon[35292]: pgmap v938: 129 pgs: 129 active+clean; 440 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.2 KiB/s rd, 2.8 MiB/s wr, 143 op/s 2024-09-19T06:40:47.866 INFO:tasks.workunit.client.0.smithi053.stdout: 10 320367 26.97 MB/sec execute 511 sec latency 502.473 ms 2024-09-19T06:40:48.049 INFO:tasks.workunit.client.1.smithi155.stdout: 10 83739 7.06 MB/sec execute 514 sec latency 90.075 ms 2024-09-19T06:40:48.866 INFO:tasks.workunit.client.0.smithi053.stdout: 10 321813 27.05 MB/sec execute 512 sec latency 108.872 ms 2024-09-19T06:40:49.049 INFO:tasks.workunit.client.1.smithi155.stdout: 10 83929 7.05 MB/sec execute 515 sec latency 73.235 ms 2024-09-19T06:40:49.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:48 smithi053 ceph-mon[27968]: pgmap v939: 129 pgs: 129 active+clean; 439 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.2 KiB/s rd, 2.7 MiB/s wr, 148 op/s 2024-09-19T06:40:49.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:48 smithi155 ceph-mon[35292]: pgmap v939: 129 pgs: 129 active+clean; 439 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.2 KiB/s rd, 2.7 MiB/s wr, 148 op/s 2024-09-19T06:40:49.866 INFO:tasks.workunit.client.0.smithi053.stdout: 10 322760 27.13 MB/sec execute 513 sec latency 127.718 ms 2024-09-19T06:40:50.049 INFO:tasks.workunit.client.1.smithi155.stdout: 10 84101 7.06 MB/sec execute 516 sec latency 94.567 ms 2024-09-19T06:40:50.866 INFO:tasks.workunit.client.0.smithi053.stdout: 10 323556 27.16 MB/sec execute 514 sec latency 99.983 ms 2024-09-19T06:40:51.049 INFO:tasks.workunit.client.1.smithi155.stdout: 10 84230 7.06 MB/sec execute 517 sec latency 83.319 ms 2024-09-19T06:40:51.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:50 smithi053 ceph-mon[27968]: pgmap v940: 129 pgs: 129 active+clean; 443 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.9 KiB/s rd, 2.9 MiB/s wr, 192 op/s 2024-09-19T06:40:51.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:50 smithi155 ceph-mon[35292]: pgmap v940: 129 pgs: 129 active+clean; 443 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.9 KiB/s rd, 2.9 MiB/s wr, 192 op/s 2024-09-19T06:40:51.866 INFO:tasks.workunit.client.0.smithi053.stdout: 10 323965 27.14 MB/sec execute 515 sec latency 96.341 ms 2024-09-19T06:40:52.049 INFO:tasks.workunit.client.1.smithi155.stdout: 10 84374 7.07 MB/sec execute 518 sec latency 84.037 ms 2024-09-19T06:40:52.866 INFO:tasks.workunit.client.0.smithi053.stdout: 10 325361 27.21 MB/sec execute 516 sec latency 230.419 ms 2024-09-19T06:40:53.049 INFO:tasks.workunit.client.1.smithi155.stdout: 10 84572 7.09 MB/sec execute 519 sec latency 119.264 ms 2024-09-19T06:40:53.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:52 smithi053 ceph-mon[27968]: pgmap v941: 129 pgs: 129 active+clean; 444 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.7 KiB/s rd, 2.2 MiB/s wr, 158 op/s 2024-09-19T06:40:53.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:52 smithi155 ceph-mon[35292]: pgmap v941: 129 pgs: 129 active+clean; 444 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.7 KiB/s rd, 2.2 MiB/s wr, 158 op/s 2024-09-19T06:40:53.866 INFO:tasks.workunit.client.0.smithi053.stdout: 10 326306 27.29 MB/sec execute 517 sec latency 120.034 ms 2024-09-19T06:40:54.049 INFO:tasks.workunit.client.1.smithi155.stdout: 10 84842 7.12 MB/sec execute 520 sec latency 83.556 ms 2024-09-19T06:40:54.866 INFO:tasks.workunit.client.0.smithi053.stdout: 10 327133 27.32 MB/sec execute 518 sec latency 104.147 ms 2024-09-19T06:40:55.049 INFO:tasks.workunit.client.1.smithi155.stdout: 10 85021 7.12 MB/sec execute 521 sec latency 155.946 ms 2024-09-19T06:40:55.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:54 smithi053 ceph-mon[27968]: pgmap v942: 129 pgs: 129 active+clean; 450 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.8 KiB/s rd, 3.5 MiB/s wr, 208 op/s 2024-09-19T06:40:55.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:54 smithi155 ceph-mon[35292]: pgmap v942: 129 pgs: 129 active+clean; 450 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.8 KiB/s rd, 3.5 MiB/s wr, 208 op/s 2024-09-19T06:40:55.866 INFO:tasks.workunit.client.0.smithi053.stdout: 10 327871 27.35 MB/sec execute 519 sec latency 114.448 ms 2024-09-19T06:40:56.049 INFO:tasks.workunit.client.1.smithi155.stdout: 10 85088 7.12 MB/sec execute 522 sec latency 98.333 ms 2024-09-19T06:40:56.866 INFO:tasks.workunit.client.0.smithi053.stdout: 10 329136 27.42 MB/sec execute 520 sec latency 118.775 ms 2024-09-19T06:40:57.049 INFO:tasks.workunit.client.1.smithi155.stdout: 10 85156 7.11 MB/sec execute 523 sec latency 113.451 ms 2024-09-19T06:40:57.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:56 smithi053 ceph-mon[27968]: pgmap v943: 129 pgs: 129 active+clean; 449 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.4 MiB/s wr, 143 op/s 2024-09-19T06:40:57.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:56 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:40:57.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:56 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:40:57.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:56 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:40:57.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:56 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:40:57.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:56 smithi155 ceph-mon[35292]: pgmap v943: 129 pgs: 129 active+clean; 449 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.4 MiB/s wr, 143 op/s 2024-09-19T06:40:57.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:56 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:40:57.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:56 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:40:57.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:56 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:40:57.233 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:56 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:40:57.866 INFO:tasks.workunit.client.0.smithi053.stdout: 10 330565 27.52 MB/sec execute 521 sec latency 117.276 ms 2024-09-19T06:40:58.049 INFO:tasks.workunit.client.1.smithi155.stdout: 10 85290 7.12 MB/sec execute 524 sec latency 95.070 ms 2024-09-19T06:40:58.866 INFO:tasks.workunit.client.0.smithi053.stdout: 10 330691 27.47 MB/sec execute 522 sec latency 94.332 ms 2024-09-19T06:40:59.049 INFO:tasks.workunit.client.1.smithi155.stdout: 10 85468 7.14 MB/sec execute 525 sec latency 93.627 ms 2024-09-19T06:40:59.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:58 smithi053 ceph-mon[27968]: pgmap v944: 129 pgs: 129 active+clean; 445 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.5 MiB/s wr, 136 op/s 2024-09-19T06:40:59.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:40:58 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:40:59.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:58 smithi155 ceph-mon[35292]: pgmap v944: 129 pgs: 129 active+clean; 445 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.5 MiB/s wr, 136 op/s 2024-09-19T06:40:59.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:40:58 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:40:59.866 INFO:tasks.workunit.client.0.smithi053.stdout: 10 331559 27.51 MB/sec execute 523 sec latency 108.303 ms 2024-09-19T06:41:00.049 INFO:tasks.workunit.client.1.smithi155.stdout: 10 85617 7.14 MB/sec execute 526 sec latency 124.313 ms 2024-09-19T06:41:00.867 INFO:tasks.workunit.client.0.smithi053.stdout: 10 332215 27.49 MB/sec execute 524 sec latency 348.911 ms 2024-09-19T06:41:01.049 INFO:tasks.workunit.client.1.smithi155.stdout: 10 85698 7.13 MB/sec execute 527 sec latency 226.295 ms 2024-09-19T06:41:01.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:00 smithi053 ceph-mon[27968]: pgmap v945: 129 pgs: 129 active+clean; 444 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.8 KiB/s rd, 4.3 MiB/s wr, 164 op/s 2024-09-19T06:41:01.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:00 smithi155 ceph-mon[35292]: pgmap v945: 129 pgs: 129 active+clean; 444 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.8 KiB/s rd, 4.3 MiB/s wr, 164 op/s 2024-09-19T06:41:01.867 INFO:tasks.workunit.client.0.smithi053.stdout: 10 333176 27.58 MB/sec execute 525 sec latency 286.052 ms 2024-09-19T06:41:02.049 INFO:tasks.workunit.client.1.smithi155.stdout: 10 85814 7.12 MB/sec execute 528 sec latency 109.727 ms 2024-09-19T06:41:02.867 INFO:tasks.workunit.client.0.smithi053.stdout: 10 334164 27.63 MB/sec execute 526 sec latency 201.342 ms 2024-09-19T06:41:03.049 INFO:tasks.workunit.client.1.smithi155.stdout: 10 85932 7.11 MB/sec execute 529 sec latency 115.264 ms 2024-09-19T06:41:03.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:02 smithi053 ceph-mon[27968]: pgmap v946: 129 pgs: 129 active+clean; 445 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 4.2 KiB/s rd, 3.9 MiB/s wr, 118 op/s 2024-09-19T06:41:03.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:02 smithi155 ceph-mon[35292]: pgmap v946: 129 pgs: 129 active+clean; 445 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 4.2 KiB/s rd, 3.9 MiB/s wr, 118 op/s 2024-09-19T06:41:03.867 INFO:tasks.workunit.client.0.smithi053.stdout: 10 334361 27.58 MB/sec execute 527 sec latency 97.070 ms 2024-09-19T06:41:04.049 INFO:tasks.workunit.client.1.smithi155.stdout: 10 86080 7.10 MB/sec execute 530 sec latency 115.366 ms 2024-09-19T06:41:04.867 INFO:tasks.workunit.client.0.smithi053.stdout: 10 335749 27.64 MB/sec execute 528 sec latency 120.427 ms 2024-09-19T06:41:05.049 INFO:tasks.workunit.client.1.smithi155.stdout: 10 86225 7.10 MB/sec execute 531 sec latency 195.700 ms 2024-09-19T06:41:05.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:04 smithi053 ceph-mon[27968]: pgmap v947: 129 pgs: 129 active+clean; 447 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.2 KiB/s rd, 5.1 MiB/s wr, 163 op/s 2024-09-19T06:41:05.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:04 smithi155 ceph-mon[35292]: pgmap v947: 129 pgs: 129 active+clean; 447 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.2 KiB/s rd, 5.1 MiB/s wr, 163 op/s 2024-09-19T06:41:05.867 INFO:tasks.workunit.client.0.smithi053.stdout: 10 336720 27.73 MB/sec execute 529 sec latency 131.238 ms 2024-09-19T06:41:06.049 INFO:tasks.workunit.client.1.smithi155.stdout: 10 86367 7.09 MB/sec execute 532 sec latency 141.106 ms 2024-09-19T06:41:06.867 INFO:tasks.workunit.client.0.smithi053.stdout: 10 336940 27.70 MB/sec execute 530 sec latency 434.784 ms 2024-09-19T06:41:07.050 INFO:tasks.workunit.client.1.smithi155.stdout: 10 86472 7.08 MB/sec execute 533 sec latency 209.161 ms 2024-09-19T06:41:07.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:06 smithi053 ceph-mon[27968]: pgmap v948: 129 pgs: 129 active+clean; 447 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.2 KiB/s rd, 3.9 MiB/s wr, 116 op/s 2024-09-19T06:41:07.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:06 smithi155 ceph-mon[35292]: pgmap v948: 129 pgs: 129 active+clean; 447 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.2 KiB/s rd, 3.9 MiB/s wr, 116 op/s 2024-09-19T06:41:07.867 INFO:tasks.workunit.client.0.smithi053.stdout: 10 337641 27.72 MB/sec execute 531 sec latency 444.510 ms 2024-09-19T06:41:08.050 INFO:tasks.workunit.client.1.smithi155.stdout: 10 86597 7.10 MB/sec execute 534 sec latency 191.039 ms 2024-09-19T06:41:08.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:08 smithi053 ceph-mon[27968]: pgmap v949: 129 pgs: 129 active+clean; 441 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.2 KiB/s rd, 3.6 MiB/s wr, 122 op/s 2024-09-19T06:41:08.867 INFO:tasks.workunit.client.0.smithi053.stdout: 10 337665 27.67 MB/sec execute 532 sec latency 436.013 ms 2024-09-19T06:41:09.050 INFO:tasks.workunit.client.1.smithi155.stdout: 10 86718 7.09 MB/sec execute 535 sec latency 243.417 ms 2024-09-19T06:41:09.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:08 smithi155 ceph-mon[35292]: pgmap v949: 129 pgs: 129 active+clean; 441 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.2 KiB/s rd, 3.6 MiB/s wr, 122 op/s 2024-09-19T06:41:09.867 INFO:tasks.workunit.client.0.smithi053.stdout: 10 337705 27.62 MB/sec execute 533 sec latency 349.277 ms 2024-09-19T06:41:10.050 INFO:tasks.workunit.client.1.smithi155.stdout: 10 86829 7.09 MB/sec execute 536 sec latency 164.162 ms 2024-09-19T06:41:10.867 INFO:tasks.workunit.client.0.smithi053.stdout: 10 337715 27.57 MB/sec execute 534 sec latency 392.685 ms 2024-09-19T06:41:11.050 INFO:tasks.workunit.client.1.smithi155.stdout: 10 86959 7.09 MB/sec execute 537 sec latency 144.989 ms 2024-09-19T06:41:11.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:10 smithi053 ceph-mon[27968]: pgmap v950: 129 pgs: 129 active+clean; 431 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 9.3 KiB/s rd, 4.0 MiB/s wr, 186 op/s 2024-09-19T06:41:11.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:10 smithi155 ceph-mon[35292]: pgmap v950: 129 pgs: 129 active+clean; 431 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 9.3 KiB/s rd, 4.0 MiB/s wr, 186 op/s 2024-09-19T06:41:11.867 INFO:tasks.workunit.client.0.smithi053.stdout: 10 337732 27.52 MB/sec execute 535 sec latency 450.685 ms 2024-09-19T06:41:12.050 INFO:tasks.workunit.client.1.smithi155.stdout: 10 87103 7.09 MB/sec execute 538 sec latency 148.133 ms 2024-09-19T06:41:12.867 INFO:tasks.workunit.client.0.smithi053.stdout: 10 337780 27.47 MB/sec execute 536 sec latency 369.768 ms 2024-09-19T06:41:13.050 INFO:tasks.workunit.client.1.smithi155.stdout: 10 87227 7.09 MB/sec execute 539 sec latency 189.379 ms 2024-09-19T06:41:13.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:12 smithi053 ceph-mon[27968]: pgmap v951: 129 pgs: 129 active+clean; 431 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.2 KiB/s rd, 3.0 MiB/s wr, 154 op/s 2024-09-19T06:41:13.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:12 smithi155 ceph-mon[35292]: pgmap v951: 129 pgs: 129 active+clean; 431 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.2 KiB/s rd, 3.0 MiB/s wr, 154 op/s 2024-09-19T06:41:13.867 INFO:tasks.workunit.client.0.smithi053.stdout: 10 337800 27.42 MB/sec execute 537 sec latency 467.283 ms 2024-09-19T06:41:14.050 INFO:tasks.workunit.client.1.smithi155.stdout: 10 87405 7.08 MB/sec execute 540 sec latency 181.306 ms 2024-09-19T06:41:14.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:13 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:41:14.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:13 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:41:14.867 INFO:tasks.workunit.client.0.smithi053.stdout: 10 337861 27.37 MB/sec execute 538 sec latency 401.195 ms 2024-09-19T06:41:15.050 INFO:tasks.workunit.client.1.smithi155.stdout: 10 87550 7.08 MB/sec execute 541 sec latency 161.601 ms 2024-09-19T06:41:15.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:14 smithi053 ceph-mon[27968]: pgmap v952: 129 pgs: 129 active+clean; 434 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.8 KiB/s rd, 3.5 MiB/s wr, 200 op/s 2024-09-19T06:41:15.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:14 smithi155 ceph-mon[35292]: pgmap v952: 129 pgs: 129 active+clean; 434 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.8 KiB/s rd, 3.5 MiB/s wr, 200 op/s 2024-09-19T06:41:15.867 INFO:tasks.workunit.client.0.smithi053.stdout: 10 337911 27.32 MB/sec execute 539 sec latency 398.739 ms 2024-09-19T06:41:16.050 INFO:tasks.workunit.client.1.smithi155.stdout: 10 87625 7.07 MB/sec execute 542 sec latency 173.335 ms 2024-09-19T06:41:16.867 INFO:tasks.workunit.client.0.smithi053.stdout: 10 338098 27.29 MB/sec execute 540 sec latency 388.718 ms 2024-09-19T06:41:17.050 INFO:tasks.workunit.client.1.smithi155.stdout: 10 87746 7.07 MB/sec execute 543 sec latency 169.072 ms 2024-09-19T06:41:17.105 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:16 smithi053 ceph-mon[27968]: pgmap v953: 129 pgs: 129 active+clean; 435 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.7 KiB/s rd, 2.3 MiB/s wr, 157 op/s 2024-09-19T06:41:17.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:16 smithi155 ceph-mon[35292]: pgmap v953: 129 pgs: 129 active+clean; 435 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.7 KiB/s rd, 2.3 MiB/s wr, 157 op/s 2024-09-19T06:41:17.867 INFO:tasks.workunit.client.0.smithi053.stdout: 10 338651 27.30 MB/sec execute 541 sec latency 509.930 ms 2024-09-19T06:41:18.050 INFO:tasks.workunit.client.1.smithi155.stdout: 10 87834 7.07 MB/sec execute 544 sec latency 172.697 ms 2024-09-19T06:41:18.868 INFO:tasks.workunit.client.0.smithi053.stdout: 10 338891 27.26 MB/sec execute 542 sec latency 551.678 ms 2024-09-19T06:41:19.050 INFO:tasks.workunit.client.1.smithi155.stdout: 10 87971 7.08 MB/sec execute 545 sec latency 151.695 ms 2024-09-19T06:41:19.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:18 smithi053 ceph-mon[27968]: pgmap v954: 129 pgs: 129 active+clean; 436 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.7 KiB/s rd, 2.3 MiB/s wr, 160 op/s 2024-09-19T06:41:19.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:18 smithi155 ceph-mon[35292]: pgmap v954: 129 pgs: 129 active+clean; 436 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.7 KiB/s rd, 2.3 MiB/s wr, 160 op/s 2024-09-19T06:41:19.867 INFO:tasks.workunit.client.0.smithi053.stdout: 10 338972 27.21 MB/sec execute 543 sec latency 516.252 ms 2024-09-19T06:41:20.050 INFO:tasks.workunit.client.1.smithi155.stdout: 10 88110 7.09 MB/sec execute 546 sec latency 264.277 ms 2024-09-19T06:41:20.867 INFO:tasks.workunit.client.0.smithi053.stdout: 10 339429 27.20 MB/sec execute 544 sec latency 611.851 ms 2024-09-19T06:41:21.050 INFO:tasks.workunit.client.1.smithi155.stdout: 10 88298 7.10 MB/sec execute 547 sec latency 177.048 ms 2024-09-19T06:41:21.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:20 smithi053 ceph-mon[27968]: pgmap v955: 129 pgs: 129 active+clean; 441 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.2 KiB/s rd, 2.8 MiB/s wr, 181 op/s 2024-09-19T06:41:21.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:20 smithi155 ceph-mon[35292]: pgmap v955: 129 pgs: 129 active+clean; 441 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.2 KiB/s rd, 2.8 MiB/s wr, 181 op/s 2024-09-19T06:41:21.867 INFO:tasks.workunit.client.0.smithi053.stdout: 10 339699 27.20 MB/sec execute 545 sec latency 514.578 ms 2024-09-19T06:41:22.050 INFO:tasks.workunit.client.1.smithi155.stdout: 10 88440 7.11 MB/sec execute 548 sec latency 212.183 ms 2024-09-19T06:41:22.867 INFO:tasks.workunit.client.0.smithi053.stdout: 10 339981 27.19 MB/sec execute 546 sec latency 651.626 ms 2024-09-19T06:41:23.050 INFO:tasks.workunit.client.1.smithi155.stdout: 10 88543 7.10 MB/sec execute 549 sec latency 248.630 ms 2024-09-19T06:41:23.104 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:22 smithi053 ceph-mon[27968]: pgmap v956: 129 pgs: 129 active+clean; 441 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.1 KiB/s rd, 2.3 MiB/s wr, 115 op/s 2024-09-19T06:41:23.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:22 smithi155 ceph-mon[35292]: pgmap v956: 129 pgs: 129 active+clean; 441 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.1 KiB/s rd, 2.3 MiB/s wr, 115 op/s 2024-09-19T06:41:23.868 INFO:tasks.workunit.client.0.smithi053.stdout: 10 340486 27.19 MB/sec execute 547 sec latency 442.815 ms 2024-09-19T06:41:24.050 INFO:tasks.workunit.client.1.smithi155.stdout: 10 88638 7.10 MB/sec execute 550 sec latency 88.646 ms 2024-09-19T06:41:24.868 INFO:tasks.workunit.client.0.smithi053.stdout: 10 341303 27.22 MB/sec execute 548 sec latency 105.853 ms 2024-09-19T06:41:25.050 INFO:tasks.workunit.client.1.smithi155.stdout: 10 88715 7.10 MB/sec execute 551 sec latency 89.927 ms 2024-09-19T06:41:25.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:24 smithi155 ceph-mon[35292]: pgmap v957: 129 pgs: 129 active+clean; 453 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 3.4 MiB/s wr, 144 op/s 2024-09-19T06:41:25.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:24 smithi053 ceph-mon[27968]: pgmap v957: 129 pgs: 129 active+clean; 453 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 3.4 MiB/s wr, 144 op/s 2024-09-19T06:41:25.868 INFO:tasks.workunit.client.0.smithi053.stdout: 10 341639 27.19 MB/sec execute 549 sec latency 261.656 ms 2024-09-19T06:41:26.050 INFO:tasks.workunit.client.1.smithi155.stdout: 10 88852 7.10 MB/sec execute 552 sec latency 102.286 ms 2024-09-19T06:41:26.868 INFO:tasks.workunit.client.0.smithi053.stdout: 10 342994 27.25 MB/sec execute 550 sec latency 120.128 ms 2024-09-19T06:41:27.050 INFO:tasks.workunit.client.1.smithi155.stdout: 10 89007 7.12 MB/sec execute 553 sec latency 140.856 ms 2024-09-19T06:41:27.232 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:26 smithi155 ceph-mon[35292]: pgmap v958: 129 pgs: 129 active+clean; 455 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.4 KiB/s rd, 3.0 MiB/s wr, 96 op/s 2024-09-19T06:41:27.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:26 smithi053 ceph-mon[27968]: pgmap v958: 129 pgs: 129 active+clean; 455 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.4 KiB/s rd, 3.0 MiB/s wr, 96 op/s 2024-09-19T06:41:27.868 INFO:tasks.workunit.client.0.smithi053.stdout: 10 344020 27.34 MB/sec execute 551 sec latency 166.877 ms 2024-09-19T06:41:28.050 INFO:tasks.workunit.client.1.smithi155.stdout: 10 89156 7.12 MB/sec execute 554 sec latency 110.048 ms 2024-09-19T06:41:28.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:28 smithi053 ceph-mon[27968]: pgmap v959: 129 pgs: 129 active+clean; 460 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.4 KiB/s rd, 3.7 MiB/s wr, 120 op/s 2024-09-19T06:41:28.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:28 smithi155 ceph-mon[35292]: pgmap v959: 129 pgs: 129 active+clean; 460 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.4 KiB/s rd, 3.7 MiB/s wr, 120 op/s 2024-09-19T06:41:28.868 INFO:tasks.workunit.client.0.smithi053.stdout: 10 344757 27.36 MB/sec execute 552 sec latency 375.446 ms 2024-09-19T06:41:29.050 INFO:tasks.workunit.client.1.smithi155.stdout: 10 89253 7.12 MB/sec execute 555 sec latency 207.910 ms 2024-09-19T06:41:29.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:41:29.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:29 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:41:29.868 INFO:tasks.workunit.client.0.smithi053.stdout: 10 344790 27.31 MB/sec execute 553 sec latency 408.962 ms 2024-09-19T06:41:30.050 INFO:tasks.workunit.client.1.smithi155.stdout: 10 89327 7.10 MB/sec execute 556 sec latency 172.850 ms 2024-09-19T06:41:30.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:30 smithi053 ceph-mon[27968]: pgmap v960: 129 pgs: 129 active+clean; 459 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.2 KiB/s rd, 4.1 MiB/s wr, 128 op/s 2024-09-19T06:41:30.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:30 smithi155 ceph-mon[35292]: pgmap v960: 129 pgs: 129 active+clean; 459 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.2 KiB/s rd, 4.1 MiB/s wr, 128 op/s 2024-09-19T06:41:30.868 INFO:tasks.workunit.client.0.smithi053.stdout: 10 344807 27.26 MB/sec execute 554 sec latency 400.266 ms 2024-09-19T06:41:31.050 INFO:tasks.workunit.client.1.smithi155.stdout: 10 89416 7.10 MB/sec execute 557 sec latency 164.054 ms 2024-09-19T06:41:31.868 INFO:tasks.workunit.client.0.smithi053.stdout: 10 344823 27.22 MB/sec execute 555 sec latency 436.522 ms 2024-09-19T06:41:32.050 INFO:tasks.workunit.client.1.smithi155.stdout: 10 89498 7.09 MB/sec execute 558 sec latency 233.773 ms 2024-09-19T06:41:32.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:32 smithi155 ceph-mon[35292]: pgmap v961: 129 pgs: 129 active+clean; 460 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.8 MiB/s wr, 100 op/s 2024-09-19T06:41:32.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:32 smithi053 ceph-mon[27968]: pgmap v961: 129 pgs: 129 active+clean; 460 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.8 MiB/s wr, 100 op/s 2024-09-19T06:41:32.868 INFO:tasks.workunit.client.0.smithi053.stdout: 10 344872 27.17 MB/sec execute 556 sec latency 387.891 ms 2024-09-19T06:41:33.051 INFO:tasks.workunit.client.1.smithi155.stdout: 10 89600 7.08 MB/sec execute 559 sec latency 159.475 ms 2024-09-19T06:41:33.868 INFO:tasks.workunit.client.0.smithi053.stdout: 10 344892 27.12 MB/sec execute 557 sec latency 406.460 ms 2024-09-19T06:41:34.051 INFO:tasks.workunit.client.1.smithi155.stdout: 10 89693 7.07 MB/sec execute 560 sec latency 238.308 ms 2024-09-19T06:41:34.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:34 smithi155 ceph-mon[35292]: pgmap v962: 129 pgs: 129 active+clean; 458 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 8.1 KiB/s rd, 4.0 MiB/s wr, 123 op/s 2024-09-19T06:41:34.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:34 smithi053 ceph-mon[27968]: pgmap v962: 129 pgs: 129 active+clean; 458 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 8.1 KiB/s rd, 4.0 MiB/s wr, 123 op/s 2024-09-19T06:41:34.868 INFO:tasks.workunit.client.0.smithi053.stdout: 10 344961 27.08 MB/sec execute 558 sec latency 378.374 ms 2024-09-19T06:41:35.051 INFO:tasks.workunit.client.1.smithi155.stdout: 10 89793 7.06 MB/sec execute 561 sec latency 176.201 ms 2024-09-19T06:41:35.868 INFO:tasks.workunit.client.0.smithi053.stdout: 10 345008 27.03 MB/sec execute 559 sec latency 394.345 ms 2024-09-19T06:41:36.051 INFO:tasks.workunit.client.1.smithi155.stdout: 10 89896 7.05 MB/sec execute 562 sec latency 203.978 ms 2024-09-19T06:41:36.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:36 smithi155 ceph-mon[35292]: pgmap v963: 129 pgs: 129 active+clean; 460 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 6.2 KiB/s rd, 3.1 MiB/s wr, 113 op/s 2024-09-19T06:41:36.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:36 smithi053 ceph-mon[27968]: pgmap v963: 129 pgs: 129 active+clean; 460 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 6.2 KiB/s rd, 3.1 MiB/s wr, 113 op/s 2024-09-19T06:41:36.868 INFO:tasks.workunit.client.0.smithi053.stdout: 10 345437 27.04 MB/sec execute 560 sec latency 434.792 ms 2024-09-19T06:41:37.051 INFO:tasks.workunit.client.1.smithi155.stdout: 10 90010 7.05 MB/sec execute 563 sec latency 186.031 ms 2024-09-19T06:41:37.868 INFO:tasks.workunit.client.0.smithi053.stdout: 10 345757 27.01 MB/sec execute 561 sec latency 550.958 ms 2024-09-19T06:41:38.051 INFO:tasks.workunit.client.1.smithi155.stdout: 10 90150 7.06 MB/sec execute 564 sec latency 165.944 ms 2024-09-19T06:41:38.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:38 smithi155 ceph-mon[35292]: pgmap v964: 129 pgs: 129 active+clean; 443 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.2 KiB/s rd, 3.0 MiB/s wr, 123 op/s 2024-09-19T06:41:38.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:38 smithi053 ceph-mon[27968]: pgmap v964: 129 pgs: 129 active+clean; 443 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.2 KiB/s rd, 3.0 MiB/s wr, 123 op/s 2024-09-19T06:41:38.868 INFO:tasks.workunit.client.0.smithi053.stdout: 10 346349 26.99 MB/sec execute 562 sec latency 416.578 ms 2024-09-19T06:41:39.052 INFO:tasks.workunit.client.1.smithi155.stdout: 10 90292 7.06 MB/sec execute 565 sec latency 134.545 ms 2024-09-19T06:41:39.868 INFO:tasks.workunit.client.0.smithi053.stdout: 10 347382 27.08 MB/sec execute 563 sec latency 122.811 ms 2024-09-19T06:41:40.051 INFO:tasks.workunit.client.1.smithi155.stdout: 10 90460 7.07 MB/sec execute 566 sec latency 95.540 ms 2024-09-19T06:41:40.484 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:40 smithi155 ceph-mon[35292]: pgmap v965: 129 pgs: 129 active+clean; 422 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.8 KiB/s rd, 2.7 MiB/s wr, 141 op/s 2024-09-19T06:41:40.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:40 smithi053 ceph-mon[27968]: pgmap v965: 129 pgs: 129 active+clean; 422 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.8 KiB/s rd, 2.7 MiB/s wr, 141 op/s 2024-09-19T06:41:40.868 INFO:tasks.workunit.client.0.smithi053.stdout: 10 348345 27.12 MB/sec execute 564 sec latency 202.802 ms 2024-09-19T06:41:41.051 INFO:tasks.workunit.client.1.smithi155.stdout: 10 90653 7.07 MB/sec execute 567 sec latency 98.575 ms 2024-09-19T06:41:41.868 INFO:tasks.workunit.client.0.smithi053.stdout: 10 348502 27.08 MB/sec execute 565 sec latency 117.031 ms 2024-09-19T06:41:42.051 INFO:tasks.workunit.client.1.smithi155.stdout: 10 90872 7.07 MB/sec execute 568 sec latency 79.032 ms 2024-09-19T06:41:42.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:42 smithi155 ceph-mon[35292]: pgmap v966: 129 pgs: 129 active+clean; 421 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.0 KiB/s rd, 2.3 MiB/s wr, 132 op/s 2024-09-19T06:41:42.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:42 smithi053 ceph-mon[27968]: pgmap v966: 129 pgs: 129 active+clean; 421 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.0 KiB/s rd, 2.3 MiB/s wr, 132 op/s 2024-09-19T06:41:42.868 INFO:tasks.workunit.client.0.smithi053.stdout: 10 349592 27.12 MB/sec execute 566 sec latency 180.801 ms 2024-09-19T06:41:43.051 INFO:tasks.workunit.client.1.smithi155.stdout: 10 91051 7.06 MB/sec execute 569 sec latency 89.009 ms 2024-09-19T06:41:43.868 INFO:tasks.workunit.client.0.smithi053.stdout: 10 350514 27.19 MB/sec execute 567 sec latency 226.561 ms 2024-09-19T06:41:44.051 INFO:tasks.workunit.client.1.smithi155.stdout: 10 91181 7.06 MB/sec execute 570 sec latency 124.539 ms 2024-09-19T06:41:44.228 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:44 smithi155 ceph-mon[35292]: pgmap v967: 129 pgs: 129 active+clean; 425 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 8.7 KiB/s rd, 2.5 MiB/s wr, 182 op/s 2024-09-19T06:41:44.228 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:44 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:41:44.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:44 smithi053 ceph-mon[27968]: pgmap v967: 129 pgs: 129 active+clean; 425 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 8.7 KiB/s rd, 2.5 MiB/s wr, 182 op/s 2024-09-19T06:41:44.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:44 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:41:44.869 INFO:tasks.workunit.client.0.smithi053.stdout: 10 350687 27.15 MB/sec execute 568 sec latency 380.248 ms 2024-09-19T06:41:45.051 INFO:tasks.workunit.client.1.smithi155.stdout: 10 91287 7.05 MB/sec execute 571 sec latency 193.603 ms 2024-09-19T06:41:45.868 INFO:tasks.workunit.client.0.smithi053.stdout: 10 350890 27.12 MB/sec execute 569 sec latency 550.687 ms 2024-09-19T06:41:46.051 INFO:tasks.workunit.client.1.smithi155.stdout: 10 91410 7.06 MB/sec execute 572 sec latency 163.459 ms 2024-09-19T06:41:46.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:46 smithi155 ceph-mon[35292]: pgmap v968: 129 pgs: 129 active+clean; 429 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 4.4 KiB/s rd, 2.4 MiB/s wr, 173 op/s 2024-09-19T06:41:46.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:46 smithi053 ceph-mon[27968]: pgmap v968: 129 pgs: 129 active+clean; 429 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 4.4 KiB/s rd, 2.4 MiB/s wr, 173 op/s 2024-09-19T06:41:46.869 INFO:tasks.workunit.client.0.smithi053.stdout: 10 351110 27.10 MB/sec execute 570 sec latency 841.425 ms 2024-09-19T06:41:47.052 INFO:tasks.workunit.client.1.smithi155.stdout: 10 91532 7.07 MB/sec execute 573 sec latency 164.629 ms 2024-09-19T06:41:47.869 INFO:tasks.workunit.client.0.smithi053.stdout: 10 351660 27.10 MB/sec execute 571 sec latency 427.220 ms 2024-09-19T06:41:48.051 INFO:tasks.workunit.client.1.smithi155.stdout: 10 91690 7.08 MB/sec execute 574 sec latency 194.152 ms 2024-09-19T06:41:48.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:48 smithi155 ceph-mon[35292]: pgmap v969: 129 pgs: 129 active+clean; 430 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 4.4 KiB/s rd, 2.4 MiB/s wr, 158 op/s 2024-09-19T06:41:48.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:48 smithi053 ceph-mon[27968]: pgmap v969: 129 pgs: 129 active+clean; 430 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 4.4 KiB/s rd, 2.4 MiB/s wr, 158 op/s 2024-09-19T06:41:48.869 INFO:tasks.workunit.client.0.smithi053.stdout: 10 351841 27.07 MB/sec execute 572 sec latency 516.705 ms 2024-09-19T06:41:49.051 INFO:tasks.workunit.client.1.smithi155.stdout: 10 91826 7.08 MB/sec execute 575 sec latency 221.181 ms 2024-09-19T06:41:49.869 INFO:tasks.workunit.client.0.smithi053.stdout: 10 351909 27.03 MB/sec execute 573 sec latency 402.341 ms 2024-09-19T06:41:50.051 INFO:tasks.workunit.client.1.smithi155.stdout: 10 92012 7.10 MB/sec execute 576 sec latency 150.548 ms 2024-09-19T06:41:50.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:50 smithi053 ceph-mon[27968]: pgmap v970: 129 pgs: 129 active+clean; 437 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.5 KiB/s rd, 3.4 MiB/s wr, 164 op/s 2024-09-19T06:41:50.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:50 smithi155 ceph-mon[35292]: pgmap v970: 129 pgs: 129 active+clean; 437 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.5 KiB/s rd, 3.4 MiB/s wr, 164 op/s 2024-09-19T06:41:50.869 INFO:tasks.workunit.client.0.smithi053.stdout: 10 352278 27.01 MB/sec execute 574 sec latency 95.239 ms 2024-09-19T06:41:51.051 INFO:tasks.workunit.client.1.smithi155.stdout: 10 92172 7.10 MB/sec execute 577 sec latency 93.356 ms 2024-09-19T06:41:51.869 INFO:tasks.workunit.client.0.smithi053.stdout: 10 353148 27.03 MB/sec execute 575 sec latency 379.361 ms 2024-09-19T06:41:52.051 INFO:tasks.workunit.client.1.smithi155.stdout: 10 92227 7.09 MB/sec execute 578 sec latency 88.073 ms 2024-09-19T06:41:52.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:52 smithi053 ceph-mon[27968]: pgmap v971: 129 pgs: 129 active+clean; 441 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.8 KiB/s rd, 3.4 MiB/s wr, 139 op/s 2024-09-19T06:41:52.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:52 smithi155 ceph-mon[35292]: pgmap v971: 129 pgs: 129 active+clean; 441 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.8 KiB/s rd, 3.4 MiB/s wr, 139 op/s 2024-09-19T06:41:52.869 INFO:tasks.workunit.client.0.smithi053.stdout: 10 354438 27.13 MB/sec execute 576 sec latency 133.434 ms 2024-09-19T06:41:53.052 INFO:tasks.workunit.client.1.smithi155.stdout: 10 92343 7.09 MB/sec execute 579 sec latency 89.387 ms 2024-09-19T06:41:53.869 INFO:tasks.workunit.client.0.smithi053.stdout: 10 355438 27.17 MB/sec execute 577 sec latency 119.169 ms 2024-09-19T06:41:54.051 INFO:tasks.workunit.client.1.smithi155.stdout: 10 92489 7.10 MB/sec execute 580 sec latency 91.876 ms 2024-09-19T06:41:54.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:54 smithi053 ceph-mon[27968]: pgmap v972: 129 pgs: 129 active+clean; 449 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 5.7 KiB/s rd, 4.0 MiB/s wr, 154 op/s 2024-09-19T06:41:54.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:54 smithi155 ceph-mon[35292]: pgmap v972: 129 pgs: 129 active+clean; 449 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 5.7 KiB/s rd, 4.0 MiB/s wr, 154 op/s 2024-09-19T06:41:54.869 INFO:tasks.workunit.client.0.smithi053.stdout: 10 355527 27.13 MB/sec execute 578 sec latency 325.566 ms 2024-09-19T06:41:55.051 INFO:tasks.workunit.client.1.smithi155.stdout: 10 92626 7.11 MB/sec execute 581 sec latency 144.705 ms 2024-09-19T06:41:55.869 INFO:tasks.workunit.client.0.smithi053.stdout: 10 355594 27.08 MB/sec execute 579 sec latency 421.475 ms 2024-09-19T06:41:56.051 INFO:tasks.workunit.client.1.smithi155.stdout: 10 92725 7.11 MB/sec execute 582 sec latency 198.895 ms 2024-09-19T06:41:56.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:56 smithi053 ceph-mon[27968]: pgmap v973: 129 pgs: 129 active+clean; 448 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.9 KiB/s rd, 3.7 MiB/s wr, 109 op/s 2024-09-19T06:41:56.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:56 smithi155 ceph-mon[35292]: pgmap v973: 129 pgs: 129 active+clean; 448 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.9 KiB/s rd, 3.7 MiB/s wr, 109 op/s 2024-09-19T06:41:56.869 INFO:tasks.workunit.client.0.smithi053.stdout: 10 355776 27.06 MB/sec execute 580 sec latency 492.492 ms 2024-09-19T06:41:57.052 INFO:tasks.workunit.client.1.smithi155.stdout: 10 92811 7.10 MB/sec execute 583 sec latency 197.083 ms 2024-09-19T06:41:57.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:57 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:57.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:57 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:41:57.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:57 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:41:57.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:57 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:57.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:57 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:41:57.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:57 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:41:57.869 INFO:tasks.workunit.client.0.smithi053.stdout: 10 356119 27.05 MB/sec execute 581 sec latency 438.082 ms 2024-09-19T06:41:58.052 INFO:tasks.workunit.client.1.smithi155.stdout: 10 92875 7.09 MB/sec execute 584 sec latency 191.500 ms 2024-09-19T06:41:58.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:58 smithi155 ceph-mon[35292]: pgmap v974: 129 pgs: 129 active+clean; 448 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.9 KiB/s rd, 3.9 MiB/s wr, 100 op/s 2024-09-19T06:41:58.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:58 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:41:58.736 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:58 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:41:58.736 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:58 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:41:58.736 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:58 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:41:58.736 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:58 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:58.736 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:58 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:58.736 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:58 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:41:58.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:58 smithi053 ceph-mon[27968]: pgmap v974: 129 pgs: 129 active+clean; 448 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.9 KiB/s rd, 3.9 MiB/s wr, 100 op/s 2024-09-19T06:41:58.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:58 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:41:58.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:58 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:41:58.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:58 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:41:58.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:58 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:41:58.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:58 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:58.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:58 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:58.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:58 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' 2024-09-19T06:41:58.869 INFO:tasks.workunit.client.0.smithi053.stdout: 10 356375 27.02 MB/sec execute 582 sec latency 598.372 ms 2024-09-19T06:41:59.052 INFO:tasks.workunit.client.1.smithi155.stdout: 10 92959 7.09 MB/sec execute 585 sec latency 163.846 ms 2024-09-19T06:41:59.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:41:59 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:41:59.755 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:41:59 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:41:59.869 INFO:tasks.workunit.client.0.smithi053.stdout: 10 356683 26.98 MB/sec execute 583 sec latency 617.871 ms 2024-09-19T06:42:00.052 INFO:tasks.workunit.client.1.smithi155.stdout: 10 93054 7.07 MB/sec execute 586 sec latency 173.561 ms 2024-09-19T06:42:00.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:00 smithi053 ceph-mon[27968]: pgmap v975: 129 pgs: 129 active+clean; 452 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 7.2 KiB/s rd, 4.4 MiB/s wr, 131 op/s 2024-09-19T06:42:00.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:00 smithi155 ceph-mon[35292]: pgmap v975: 129 pgs: 129 active+clean; 452 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 7.2 KiB/s rd, 4.4 MiB/s wr, 131 op/s 2024-09-19T06:42:00.869 INFO:tasks.workunit.client.0.smithi053.stdout: 10 357250 26.99 MB/sec execute 584 sec latency 394.057 ms 2024-09-19T06:42:01.052 INFO:tasks.workunit.client.1.smithi155.stdout: 10 93161 7.07 MB/sec execute 587 sec latency 186.645 ms 2024-09-19T06:42:01.869 INFO:tasks.workunit.client.0.smithi053.stdout: 10 358204 27.06 MB/sec execute 585 sec latency 171.524 ms 2024-09-19T06:42:02.052 INFO:tasks.workunit.client.1.smithi155.stdout: 10 93314 7.06 MB/sec execute 588 sec latency 114.582 ms 2024-09-19T06:42:02.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:02 smithi155 ceph-mon[35292]: pgmap v976: 129 pgs: 129 active+clean; 453 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.2 KiB/s rd, 3.6 MiB/s wr, 120 op/s 2024-09-19T06:42:02.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:02 smithi053 ceph-mon[27968]: pgmap v976: 129 pgs: 129 active+clean; 453 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.2 KiB/s rd, 3.6 MiB/s wr, 120 op/s 2024-09-19T06:42:02.869 INFO:tasks.workunit.client.0.smithi053.stdout: 10 358987 27.08 MB/sec execute 586 sec latency 210.622 ms 2024-09-19T06:42:03.052 INFO:tasks.workunit.client.1.smithi155.stdout: 10 93468 7.06 MB/sec execute 589 sec latency 102.668 ms 2024-09-19T06:42:03.869 INFO:tasks.workunit.client.0.smithi053.stdout: 10 359182 27.04 MB/sec execute 587 sec latency 93.747 ms 2024-09-19T06:42:04.052 INFO:tasks.workunit.client.1.smithi155.stdout: 10 93620 7.06 MB/sec execute 590 sec latency 105.521 ms 2024-09-19T06:42:04.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:04 smithi155 ceph-mon[35292]: pgmap v977: 129 pgs: 129 active+clean; 444 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 8.6 KiB/s rd, 3.3 MiB/s wr, 125 op/s 2024-09-19T06:42:04.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:04 smithi053 ceph-mon[27968]: pgmap v977: 129 pgs: 129 active+clean; 444 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 8.6 KiB/s rd, 3.3 MiB/s wr, 125 op/s 2024-09-19T06:42:04.869 INFO:tasks.workunit.client.0.smithi053.stdout: 10 360240 27.08 MB/sec execute 588 sec latency 150.923 ms 2024-09-19T06:42:05.052 INFO:tasks.workunit.client.1.smithi155.stdout: 10 93789 7.07 MB/sec execute 591 sec latency 112.772 ms 2024-09-19T06:42:05.869 INFO:tasks.workunit.client.0.smithi053.stdout: 10 361303 27.16 MB/sec execute 589 sec latency 204.008 ms 2024-09-19T06:42:06.052 INFO:tasks.workunit.client.1.smithi155.stdout: 10 93926 7.06 MB/sec execute 592 sec latency 120.975 ms 2024-09-19T06:42:06.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:06 smithi155 ceph-mon[35292]: pgmap v978: 129 pgs: 129 active+clean; 441 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.7 KiB/s rd, 2.9 MiB/s wr, 149 op/s 2024-09-19T06:42:06.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:06 smithi053 ceph-mon[27968]: pgmap v978: 129 pgs: 129 active+clean; 441 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.7 KiB/s rd, 2.9 MiB/s wr, 149 op/s 2024-09-19T06:42:06.869 INFO:tasks.workunit.client.0.smithi053.stdout: 10 361668 27.15 MB/sec execute 590 sec latency 387.842 ms 2024-09-19T06:42:07.052 INFO:tasks.workunit.client.1.smithi155.stdout: 10 94080 7.07 MB/sec execute 593 sec latency 155.204 ms 2024-09-19T06:42:07.870 INFO:tasks.workunit.client.0.smithi053.stdout: 10 362381 27.16 MB/sec execute 591 sec latency 342.772 ms 2024-09-19T06:42:08.052 INFO:tasks.workunit.client.1.smithi155.stdout: 10 94227 7.07 MB/sec execute 594 sec latency 139.541 ms 2024-09-19T06:42:08.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:08 smithi155 ceph-mon[35292]: pgmap v979: 129 pgs: 129 active+clean; 439 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.7 KiB/s rd, 2.9 MiB/s wr, 151 op/s 2024-09-19T06:42:08.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:08 smithi053 ceph-mon[27968]: pgmap v979: 129 pgs: 129 active+clean; 439 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.7 KiB/s rd, 2.9 MiB/s wr, 151 op/s 2024-09-19T06:42:08.869 INFO:tasks.workunit.client.0.smithi053.stdout: 10 362486 27.13 MB/sec execute 592 sec latency 353.895 ms 2024-09-19T06:42:09.052 INFO:tasks.workunit.client.1.smithi155.stdout: 10 94401 7.07 MB/sec execute 595 sec latency 145.510 ms 2024-09-19T06:42:09.870 INFO:tasks.workunit.client.0.smithi053.stdout: 10 362529 27.08 MB/sec execute 593 sec latency 312.964 ms 2024-09-19T06:42:10.052 INFO:tasks.workunit.client.1.smithi155.stdout: 10 94555 7.06 MB/sec execute 596 sec latency 170.242 ms 2024-09-19T06:42:10.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:10 smithi155 ceph-mon[35292]: pgmap v980: 129 pgs: 129 active+clean; 441 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 8.9 KiB/s rd, 3.1 MiB/s wr, 192 op/s 2024-09-19T06:42:10.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:10 smithi053 ceph-mon[27968]: pgmap v980: 129 pgs: 129 active+clean; 441 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 8.9 KiB/s rd, 3.1 MiB/s wr, 192 op/s 2024-09-19T06:42:10.870 INFO:tasks.workunit.client.0.smithi053.stdout: 10 362553 27.04 MB/sec execute 594 sec latency 327.959 ms 2024-09-19T06:42:11.052 INFO:tasks.workunit.client.1.smithi155.stdout: 10 94671 7.06 MB/sec execute 597 sec latency 163.432 ms 2024-09-19T06:42:11.870 INFO:tasks.workunit.client.0.smithi053.stdout: 10 362599 26.99 MB/sec execute 595 sec latency 330.803 ms 2024-09-19T06:42:12.052 INFO:tasks.workunit.client.1.smithi155.stdout: 10 94757 7.05 MB/sec execute 598 sec latency 148.961 ms 2024-09-19T06:42:12.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:12 smithi155 ceph-mon[35292]: pgmap v981: 129 pgs: 129 active+clean; 441 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 5.6 KiB/s rd, 2.4 MiB/s wr, 155 op/s 2024-09-19T06:42:12.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:12 smithi053 ceph-mon[27968]: pgmap v981: 129 pgs: 129 active+clean; 441 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 5.6 KiB/s rd, 2.4 MiB/s wr, 155 op/s 2024-09-19T06:42:12.870 INFO:tasks.workunit.client.0.smithi053.stdout: 10 362676 26.95 MB/sec execute 596 sec latency 283.523 ms 2024-09-19T06:42:13.052 INFO:tasks.workunit.client.1.smithi155.stdout: 10 94883 7.05 MB/sec execute 599 sec latency 131.087 ms 2024-09-19T06:42:13.870 INFO:tasks.workunit.client.0.smithi053.stdout: 10 362904 26.93 MB/sec execute 597 sec latency 270.529 ms 2024-09-19T06:42:14.052 INFO:tasks.workunit.client.1.smithi155.stdout: 10 cleanup 600 sec 2024-09-19T06:42:14.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:14 smithi155 ceph-mon[35292]: pgmap v982: 129 pgs: 129 active+clean; 444 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.9 KiB/s rd, 2.7 MiB/s wr, 191 op/s 2024-09-19T06:42:14.733 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:14 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:42:14.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:14 smithi053 ceph-mon[27968]: pgmap v982: 129 pgs: 129 active+clean; 444 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.9 KiB/s rd, 2.7 MiB/s wr, 191 op/s 2024-09-19T06:42:14.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:14 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:42:14.870 INFO:tasks.workunit.client.0.smithi053.stdout: 10 363480 26.94 MB/sec execute 598 sec latency 448.353 ms 2024-09-19T06:42:15.052 INFO:tasks.workunit.client.1.smithi155.stdout: 10 cleanup 601 sec 2024-09-19T06:42:15.870 INFO:tasks.workunit.client.0.smithi053.stdout: 10 363826 26.90 MB/sec execute 599 sec latency 384.656 ms 2024-09-19T06:42:16.052 INFO:tasks.workunit.client.1.smithi155.stdout: 10 cleanup 602 sec 2024-09-19T06:42:16.198 INFO:tasks.workunit.client.1.smithi155.stdout: 0 cleanup 602 sec 2024-09-19T06:42:16.198 INFO:tasks.workunit.client.1.smithi155.stdout: 2024-09-19T06:42:16.198 INFO:tasks.workunit.client.1.smithi155.stdout: Operation Count AvgLat MaxLat 2024-09-19T06:42:16.199 INFO:tasks.workunit.client.1.smithi155.stdout: ---------------------------------------- 2024-09-19T06:42:16.199 INFO:tasks.workunit.client.1.smithi155.stdout: NTCreateX 135374 10.040 278.827 2024-09-19T06:42:16.199 INFO:tasks.workunit.client.1.smithi155.stdout: Close 99451 0.115 14.046 2024-09-19T06:42:16.199 INFO:tasks.workunit.client.1.smithi155.stdout: Rename 5748 25.038 234.093 2024-09-19T06:42:16.199 INFO:tasks.workunit.client.1.smithi155.stdout: Unlink 27333 9.192 188.485 2024-09-19T06:42:16.199 INFO:tasks.workunit.client.1.smithi155.stdout: Qpathinfo 122699 5.540 272.107 2024-09-19T06:42:16.200 INFO:tasks.workunit.client.1.smithi155.stdout: Qfileinfo 21425 0.210 45.880 2024-09-19T06:42:16.200 INFO:tasks.workunit.client.1.smithi155.stdout: Qfsinfo 22537 0.578 72.825 2024-09-19T06:42:16.200 INFO:tasks.workunit.client.1.smithi155.stdout: Sfileinfo 11028 1.486 29.130 2024-09-19T06:42:16.200 INFO:tasks.workunit.client.1.smithi155.stdout: Find 47489 71.172 352.993 2024-09-19T06:42:16.200 INFO:tasks.workunit.client.1.smithi155.stdout: WriteX 66911 0.421 111.075 2024-09-19T06:42:16.201 INFO:tasks.workunit.client.1.smithi155.stdout: ReadX 213076 0.005 4.401 2024-09-19T06:42:16.201 INFO:tasks.workunit.client.1.smithi155.stdout: LockX 444 4.937 93.279 2024-09-19T06:42:16.201 INFO:tasks.workunit.client.1.smithi155.stdout: UnlockX 444 3.552 115.253 2024-09-19T06:42:16.201 INFO:tasks.workunit.client.1.smithi155.stdout: Flush 9483 11.223 191.854 2024-09-19T06:42:16.201 INFO:tasks.workunit.client.1.smithi155.stdout: 2024-09-19T06:42:16.201 INFO:tasks.workunit.client.1.smithi155.stdout:Throughput 7.05269 MB/sec 10 clients 10 procs max_latency=353.000 ms 2024-09-19T06:42:16.202 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-19T06:42:16.203 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-09-19T06:42:16.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:16 smithi155 ceph-mon[35292]: pgmap v983: 129 pgs: 129 active+clean; 446 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.5 KiB/s rd, 2.8 MiB/s wr, 176 op/s 2024-09-19T06:42:16.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:16 smithi053 ceph-mon[27968]: pgmap v983: 129 pgs: 129 active+clean; 446 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.5 KiB/s rd, 2.8 MiB/s wr, 176 op/s 2024-09-19T06:42:16.870 INFO:tasks.workunit.client.0.smithi053.stdout: 10 cleanup 600 sec 2024-09-19T06:42:16.955 INFO:tasks.workunit.client.0.smithi053.stdout: 0 cleanup 600 sec 2024-09-19T06:42:16.955 INFO:tasks.workunit.client.0.smithi053.stdout: 2024-09-19T06:42:16.955 INFO:tasks.workunit.client.0.smithi053.stdout: Operation Count AvgLat MaxLat 2024-09-19T06:42:16.955 INFO:tasks.workunit.client.0.smithi053.stdout: ---------------------------------------- 2024-09-19T06:42:16.955 INFO:tasks.workunit.client.0.smithi053.stdout: NTCreateX 518965 0.061 132.403 2024-09-19T06:42:16.955 INFO:tasks.workunit.client.0.smithi053.stdout: Close 381394 0.001 1.471 2024-09-19T06:42:16.955 INFO:tasks.workunit.client.0.smithi053.stdout: Rename 22031 0.401 120.568 2024-09-19T06:42:16.956 INFO:tasks.workunit.client.0.smithi053.stdout: Unlink 104595 0.763 130.955 2024-09-19T06:42:16.956 INFO:tasks.workunit.client.0.smithi053.stdout: Qpathinfo 471122 0.038 140.559 2024-09-19T06:42:16.956 INFO:tasks.workunit.client.0.smithi053.stdout: Qfileinfo 82292 0.001 0.112 2024-09-19T06:42:16.956 INFO:tasks.workunit.client.0.smithi053.stdout: Qfsinfo 86158 0.002 2.077 2024-09-19T06:42:16.956 INFO:tasks.workunit.client.0.smithi053.stdout: Sfileinfo 42417 2.204 160.785 2024-09-19T06:42:16.956 INFO:tasks.workunit.client.0.smithi053.stdout: Find 181888 0.024 93.902 2024-09-19T06:42:16.956 INFO:tasks.workunit.client.0.smithi053.stdout: WriteX 256740 0.461 170.094 2024-09-19T06:42:16.956 INFO:tasks.workunit.client.0.smithi053.stdout: ReadX 815550 0.033 120.214 2024-09-19T06:42:16.956 INFO:tasks.workunit.client.0.smithi053.stdout: LockX 1700 0.003 0.025 2024-09-19T06:42:16.956 INFO:tasks.workunit.client.0.smithi053.stdout: UnlockX 1700 0.002 0.017 2024-09-19T06:42:16.956 INFO:tasks.workunit.client.0.smithi053.stdout: Flush 36340 154.491 841.414 2024-09-19T06:42:16.956 INFO:tasks.workunit.client.0.smithi053.stdout: 2024-09-19T06:42:16.956 INFO:tasks.workunit.client.0.smithi053.stdout:Throughput 26.9044 MB/sec 10 clients 10 procs max_latency=841.425 ms 2024-09-19T06:42:16.957 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-19T06:42:16.957 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-19T06:42:17.179 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-09-19T06:42:17.179 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-19T06:42:17.485 DEBUG:teuthology.parallel:result is None 2024-09-19T06:42:17.912 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.1... 2024-09-19T06:42:17.912 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-09-19T06:42:18.693 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:18 smithi155 ceph-mon[35292]: pgmap v984: 129 pgs: 129 active+clean; 444 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.5 KiB/s rd, 2.6 MiB/s wr, 135 op/s 2024-09-19T06:42:18.694 DEBUG:teuthology.parallel:result is None 2024-09-19T06:42:18.695 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-19T06:42:18.721 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:18 smithi053 ceph-mon[27968]: pgmap v984: 129 pgs: 129 active+clean; 444 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.5 KiB/s rd, 2.6 MiB/s wr, 135 op/s 2024-09-19T06:42:18.725 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-19T06:42:18.726 DEBUG:teuthology.orchestra.run.smithi053:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-09-19T06:42:18.802 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-09-19T06:42:18.802 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-09-19T06:42:18.834 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-09-19T06:42:18.834 INFO:teuthology.run_tasks:Running task pexec... 2024-09-19T06:42:18.908 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-19T06:42:18.909 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi053.front.sepia.ceph.com 2024-09-19T06:42:18.909 DEBUG:teuthology.orchestra.run.smithi053:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-19T06:42:18.910 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi155.front.sepia.ceph.com 2024-09-19T06:42:18.910 DEBUG:teuthology.orchestra.run.smithi155:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-19T06:42:19.704 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-09-19T06:42:19.705 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:42:19.705 INFO:teuthology.orchestra.run.smithi155.stdout: Package Architecture Version Repository Size 2024-09-19T06:42:19.705 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:42:19.706 INFO:teuthology.orchestra.run.smithi155.stdout:Removing: 2024-09-19T06:42:19.706 INFO:teuthology.orchestra.run.smithi155.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-19T06:42:19.706 INFO:teuthology.orchestra.run.smithi155.stdout:Removing unused dependencies: 2024-09-19T06:42:19.706 INFO:teuthology.orchestra.run.smithi155.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-19T06:42:19.706 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:42:19.706 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-09-19T06:42:19.706 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:42:19.706 INFO:teuthology.orchestra.run.smithi155.stdout:Remove 2 Packages 2024-09-19T06:42:19.706 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:42:19.706 INFO:teuthology.orchestra.run.smithi155.stdout:Freed space: 5.5 M 2024-09-19T06:42:19.706 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-09-19T06:42:19.753 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-09-19T06:42:19.753 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-09-19T06:42:19.879 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-09-19T06:42:19.880 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-09-19T06:42:20.146 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-09-19T06:42:20.310 INFO:teuthology.orchestra.run.smithi155.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-19T06:42:20.406 INFO:teuthology.orchestra.run.smithi155.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-19T06:42:20.447 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-19T06:42:20.448 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:42:20.448 INFO:teuthology.orchestra.run.smithi053.stdout: Package Architecture Version Repository Size 2024-09-19T06:42:20.448 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:42:20.448 INFO:teuthology.orchestra.run.smithi053.stdout:Removing: 2024-09-19T06:42:20.448 INFO:teuthology.orchestra.run.smithi053.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-19T06:42:20.448 INFO:teuthology.orchestra.run.smithi053.stdout:Removing unused dependencies: 2024-09-19T06:42:20.448 INFO:teuthology.orchestra.run.smithi053.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-19T06:42:20.449 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:42:20.449 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-19T06:42:20.449 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:42:20.449 INFO:teuthology.orchestra.run.smithi053.stdout:Remove 2 Packages 2024-09-19T06:42:20.449 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:42:20.449 INFO:teuthology.orchestra.run.smithi053.stdout:Freed space: 5.5 M 2024-09-19T06:42:20.449 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-19T06:42:20.492 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-19T06:42:20.492 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-19T06:42:20.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:20 smithi053 ceph-mon[27968]: pgmap v985: 129 pgs: 129 active+clean; 328 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 10 KiB/s rd, 2.4 MiB/s wr, 154 op/s 2024-09-19T06:42:20.719 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-19T06:42:20.726 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-19T06:42:20.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:20 smithi155 ceph-mon[35292]: pgmap v985: 129 pgs: 129 active+clean; 328 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 10 KiB/s rd, 2.4 MiB/s wr, 154 op/s 2024-09-19T06:42:20.956 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-19T06:42:21.090 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-19T06:42:21.090 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-19T06:42:21.206 INFO:teuthology.orchestra.run.smithi053.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-19T06:42:21.561 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-19T06:42:21.562 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:42:21.562 INFO:teuthology.orchestra.run.smithi155.stdout:Removed: 2024-09-19T06:42:21.562 INFO:teuthology.orchestra.run.smithi155.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-19T06:42:21.562 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:42:21.562 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-09-19T06:42:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-19T06:42:22.458 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:35:37 ago on Thu 19 Sep 2024 06:06:45 AM UTC. 2024-09-19T06:42:22.599 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-09-19T06:42:22.600 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:42:22.600 INFO:teuthology.orchestra.run.smithi155.stdout: Package Architecture Version Repository Size 2024-09-19T06:42:22.600 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:42:22.600 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-09-19T06:42:22.600 INFO:teuthology.orchestra.run.smithi155.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-19T06:42:22.600 INFO:teuthology.orchestra.run.smithi155.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-19T06:42:22.600 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-09-19T06:42:22.601 INFO:teuthology.orchestra.run.smithi155.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-19T06:42:22.601 INFO:teuthology.orchestra.run.smithi155.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-19T06:42:22.601 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-19T06:42:22.601 INFO:teuthology.orchestra.run.smithi155.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-19T06:42:22.601 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:42:22.601 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-09-19T06:42:22.601 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-09-19T06:42:22.601 INFO:teuthology.orchestra.run.smithi155.stdout:Install 6 Packages 2024-09-19T06:42:22.601 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:42:22.610 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 2.0 M 2024-09-19T06:42:22.610 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 9.2 M 2024-09-19T06:42:22.610 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-09-19T06:42:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-19T06:42:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-19T06:42:22.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:22 smithi155 ceph-mon[35292]: pgmap v986: 129 pgs: 129 active+clean; 297 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 8.2 KiB/s rd, 2.0 MiB/s wr, 126 op/s 2024-09-19T06:42:22.836 INFO:teuthology.orchestra.run.smithi155.stdout:(1/6): nvmetcli-0.7-3.el9.noarch.rpm 467 kB/s | 44 kB 00:00 2024-09-19T06:42:22.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:22 smithi053 ceph-mon[27968]: pgmap v986: 129 pgs: 129 active+clean; 297 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 8.2 KiB/s rd, 2.0 MiB/s wr, 126 op/s 2024-09-19T06:42:22.887 INFO:teuthology.orchestra.run.smithi155.stdout:(2/6): libnvme-1.9-3.el9.x86_64.rpm 727 kB/s | 106 kB 00:00 2024-09-19T06:42:22.937 INFO:teuthology.orchestra.run.smithi155.stdout:(3/6): python3-configshell-1.1.30-1.el9.noarch. 712 kB/s | 72 kB 00:00 2024-09-19T06:42:22.980 INFO:teuthology.orchestra.run.smithi155.stdout:(4/6): python3-kmod-0.9-32.el9.x86_64.rpm 910 kB/s | 84 kB 00:00 2024-09-19T06:42:23.038 INFO:teuthology.orchestra.run.smithi155.stdout:(5/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 3.0 MB/s | 906 kB 00:00 2024-09-19T06:42:23.148 INFO:teuthology.orchestra.run.smithi155.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 3.9 MB/s | 837 kB 00:00 2024-09-19T06:42:23.148 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:42:23.148 INFO:teuthology.orchestra.run.smithi155.stdout:Total 3.7 MB/s | 2.0 MB 00:00 2024-09-19T06:42:23.179 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-19T06:42:23.179 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:42:23.179 INFO:teuthology.orchestra.run.smithi053.stdout:Removed: 2024-09-19T06:42:23.180 INFO:teuthology.orchestra.run.smithi053.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-19T06:42:23.180 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:42:23.180 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-19T06:42:23.240 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-09-19T06:42:23.275 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-09-19T06:42:23.275 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-09-19T06:42:23.607 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-09-19T06:42:23.607 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-09-19T06:42:24.096 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-09-19T06:42:24.201 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-19T06:42:24.235 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-19T06:42:24.271 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:35:50 ago on Thu 19 Sep 2024 06:06:34 AM UTC. 2024-09-19T06:42:24.272 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-19T06:42:24.412 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-19T06:42:24.413 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:42:24.413 INFO:teuthology.orchestra.run.smithi053.stdout: Package Architecture Version Repository Size 2024-09-19T06:42:24.413 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:42:24.413 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-19T06:42:24.413 INFO:teuthology.orchestra.run.smithi053.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-19T06:42:24.413 INFO:teuthology.orchestra.run.smithi053.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-19T06:42:24.413 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-19T06:42:24.413 INFO:teuthology.orchestra.run.smithi053.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-19T06:42:24.413 INFO:teuthology.orchestra.run.smithi053.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-19T06:42:24.413 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-19T06:42:24.413 INFO:teuthology.orchestra.run.smithi053.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-19T06:42:24.414 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:42:24.414 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-19T06:42:24.414 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-19T06:42:24.414 INFO:teuthology.orchestra.run.smithi053.stdout:Install 6 Packages 2024-09-19T06:42:24.414 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:42:24.455 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 2.0 M 2024-09-19T06:42:24.456 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 9.2 M 2024-09-19T06:42:24.456 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-19T06:42:24.492 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-19T06:42:24.529 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-19T06:42:24.700 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:24 smithi053 ceph-mon[27968]: pgmap v987: 129 pgs: 129 active+clean; 218 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 9.6 KiB/s rd, 1.8 MiB/s wr, 153 op/s 2024-09-19T06:42:24.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:24 smithi155 ceph-mon[35292]: pgmap v987: 129 pgs: 129 active+clean; 218 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 9.6 KiB/s rd, 1.8 MiB/s wr, 153 op/s 2024-09-19T06:42:24.899 INFO:teuthology.orchestra.run.smithi053.stdout:(1/6): nvmetcli-0.7-3.el9.noarch.rpm 380 kB/s | 44 kB 00:00 2024-09-19T06:42:24.933 INFO:teuthology.orchestra.run.smithi053.stdout:(2/6): libnvme-1.9-3.el9.x86_64.rpm 705 kB/s | 106 kB 00:00 2024-09-19T06:42:24.983 INFO:teuthology.orchestra.run.smithi053.stdout:(3/6): python3-configshell-1.1.30-1.el9.noarch. 857 kB/s | 72 kB 00:00 2024-09-19T06:42:25.017 INFO:teuthology.orchestra.run.smithi053.stdout:(4/6): python3-kmod-0.9-32.el9.x86_64.rpm 1.0 MB/s | 84 kB 00:00 2024-09-19T06:42:25.059 INFO:teuthology.orchestra.run.smithi053.stdout:(5/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 3.2 MB/s | 906 kB 00:00 2024-09-19T06:42:25.128 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-19T06:42:25.143 INFO:teuthology.orchestra.run.smithi053.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 5.1 MB/s | 837 kB 00:00 2024-09-19T06:42:25.143 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:42:25.143 INFO:teuthology.orchestra.run.smithi053.stdout:Total 2.9 MB/s | 2.0 MB 00:00 2024-09-19T06:42:25.164 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-19T06:42:25.229 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-19T06:42:25.284 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-19T06:42:25.284 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-19T06:42:25.744 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-19T06:42:25.744 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-19T06:42:26.283 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-19T06:42:26.396 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-19T06:42:26.415 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-19T06:42:26.415 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-19T06:42:26.415 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-19T06:42:26.415 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-19T06:42:26.416 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-19T06:42:26.416 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-19T06:42:26.446 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-19T06:42:26.484 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-19T06:42:26.723 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-19T06:42:26.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:26 smithi155 ceph-mon[35292]: pgmap v988: 129 pgs: 129 active+clean; 218 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 8.2 KiB/s rd, 1.2 MiB/s wr, 110 op/s 2024-09-19T06:42:26.765 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-19T06:42:26.778 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:26 smithi053 ceph-mon[27968]: pgmap v988: 129 pgs: 129 active+clean; 218 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 8.2 KiB/s rd, 1.2 MiB/s wr, 110 op/s 2024-09-19T06:42:26.810 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-19T06:42:26.810 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:42:26.810 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-09-19T06:42:26.810 INFO:teuthology.orchestra.run.smithi155.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-19T06:42:26.810 INFO:teuthology.orchestra.run.smithi155.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-19T06:42:26.810 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-19T06:42:26.810 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-19T06:42:26.810 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-09-19T06:42:27.174 DEBUG:teuthology.parallel:result is None 2024-09-19T06:42:27.419 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-19T06:42:27.458 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-19T06:42:28.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:28 smithi053 ceph-mon[27968]: pgmap v989: 129 pgs: 129 active+clean; 218 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 8.2 KiB/s rd, 783 KiB/s wr, 96 op/s 2024-09-19T06:42:28.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:28 smithi155 ceph-mon[35292]: pgmap v989: 129 pgs: 129 active+clean; 218 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 8.2 KiB/s rd, 783 KiB/s wr, 96 op/s 2024-09-19T06:42:28.800 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-19T06:42:28.800 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-19T06:42:28.800 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-19T06:42:28.800 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-19T06:42:28.800 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-19T06:42:28.800 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-19T06:42:29.266 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-19T06:42:29.266 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:42:29.266 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-19T06:42:29.267 INFO:teuthology.orchestra.run.smithi053.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-19T06:42:29.267 INFO:teuthology.orchestra.run.smithi053.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-19T06:42:29.267 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-19T06:42:29.267 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-19T06:42:29.267 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-19T06:42:29.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:29 smithi053 ceph-mon[27968]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:42:29.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:29 smithi155 ceph-mon[35292]: from='mgr.14217 172.21.15.53:0/606031466' entity='mgr.smithi053.trakvg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-19T06:42:29.701 DEBUG:teuthology.parallel:result is None 2024-09-19T06:42:29.702 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-19T06:42:29.712 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-19T06:42:29.712 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-19T06:42:29.712 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T06:42:29.743 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-19T06:42:29.744 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T06:42:29.803 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-19T06:42:30.086 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:42:30.307 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:30 smithi053 ceph-mon[27968]: pgmap v990: 129 pgs: 129 active+clean; 218 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 8.3 KiB/s rd, 460 KiB/s wr, 89 op/s 2024-09-19T06:42:30.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:30 smithi155 ceph-mon[35292]: pgmap v990: 129 pgs: 129 active+clean; 218 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 8.3 KiB/s rd, 460 KiB/s wr, 89 op/s 2024-09-19T06:42:30.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.763+0000 7fc1c2531640 1 -- 172.21.15.53:0/410922129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1bc1033d0 msgr2=0x7fc1bc10b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:30.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.763+0000 7fc1c2531640 1 --2- 172.21.15.53:0/410922129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1bc1033d0 0x7fc1bc10b8c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0009880 tx=0x7fc1b002a530 comp rx=0 tx=0).stop 2024-09-19T06:42:30.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.763+0000 7fc1c2531640 1 -- 172.21.15.53:0/410922129 shutdown_connections 2024-09-19T06:42:30.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.763+0000 7fc1c2531640 1 --2- 172.21.15.53:0/410922129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1bc1033d0 0x7fc1bc10b8c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:30.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.763+0000 7fc1c2531640 1 --2- 172.21.15.53:0/410922129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1bc102ab0 0x7fc1bc102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:30.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.763+0000 7fc1c2531640 1 -- 172.21.15.53:0/410922129 >> 172.21.15.53:0/410922129 conn(0x7fc1bc0fc550 msgr2=0x7fc1bc0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:30.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.763+0000 7fc1c2531640 1 -- 172.21.15.53:0/410922129 shutdown_connections 2024-09-19T06:42:30.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.763+0000 7fc1c2531640 1 -- 172.21.15.53:0/410922129 wait complete. 2024-09-19T06:42:30.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.764+0000 7fc1c2531640 1 Processor -- start 2024-09-19T06:42:30.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.764+0000 7fc1c2531640 1 -- start start 2024-09-19T06:42:30.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.764+0000 7fc1c2531640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1bc102ab0 0x7fc1bc100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:30.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.764+0000 7fc1c2531640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1bc1033d0 0x7fc1bc100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:30.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.764+0000 7fc1c2531640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc10ed30 con 0x7fc1bc102ab0 2024-09-19T06:42:30.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.764+0000 7fc1c2531640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc10ebb0 con 0x7fc1bc1033d0 2024-09-19T06:42:30.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.764+0000 7fc1bbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1bc102ab0 0x7fc1bc100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:30.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.764+0000 7fc1bbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1bc102ab0 0x7fc1bc100710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33054/0 (socket says 172.21.15.53:33054) 2024-09-19T06:42:30.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.764+0000 7fc1bbfff640 1 -- 172.21.15.53:0/218269170 learned_addr learned my addr 172.21.15.53:0/218269170 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:42:30.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.764+0000 7fc1bb7fe640 1 --2- 172.21.15.53:0/218269170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1bc1033d0 0x7fc1bc100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:30.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.765+0000 7fc1bbfff640 1 -- 172.21.15.53:0/218269170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1bc1033d0 msgr2=0x7fc1bc100c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:30.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.765+0000 7fc1bbfff640 1 --2- 172.21.15.53:0/218269170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1bc1033d0 0x7fc1bc100c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:30.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.765+0000 7fc1bbfff640 1 -- 172.21.15.53:0/218269170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1bc101190 con 0x7fc1bc102ab0 2024-09-19T06:42:30.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.765+0000 7fc1bbfff640 1 --2- 172.21.15.53:0/218269170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1bc102ab0 0x7fc1bc100710 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac00c810 tx=0x7fc1ac00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:30.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.765+0000 7fc1b97fa640 1 -- 172.21.15.53:0/218269170 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc1ac004020 con 0x7fc1bc102ab0 2024-09-19T06:42:30.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.765+0000 7fc1c2531640 1 -- 172.21.15.53:0/218269170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1bc101480 con 0x7fc1bc102ab0 2024-09-19T06:42:30.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.765+0000 7fc1b97fa640 1 -- 172.21.15.53:0/218269170 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1ac0041c0 con 0x7fc1bc102ab0 2024-09-19T06:42:30.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.765+0000 7fc1b97fa640 1 -- 172.21.15.53:0/218269170 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc1ac0047a0 con 0x7fc1bc102ab0 2024-09-19T06:42:30.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.765+0000 7fc1c2531640 1 -- 172.21.15.53:0/218269170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1bc1a3cb0 con 0x7fc1bc102ab0 2024-09-19T06:42:30.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.766+0000 7fc1c2531640 1 -- 172.21.15.53:0/218269170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1bc06a580 con 0x7fc1bc102ab0 2024-09-19T06:42:30.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.767+0000 7fc1b97fa640 1 -- 172.21.15.53:0/218269170 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7fc1ac004a50 con 0x7fc1bc102ab0 2024-09-19T06:42:30.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.767+0000 7fc1b97fa640 1 --2- 172.21.15.53:0/218269170 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fc1a406e2b0 0x7fc1a4070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:30.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.768+0000 7fc1bb7fe640 1 --2- 172.21.15.53:0/218269170 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fc1a406e2b0 0x7fc1a4070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:30.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.768+0000 7fc1b97fa640 1 -- 172.21.15.53:0/218269170 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7340+0+0 (secure 0 0 0) 0x7fc1ac08e790 con 0x7fc1bc102ab0 2024-09-19T06:42:30.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.768+0000 7fc1bb7fe640 1 --2- 172.21.15.53:0/218269170 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fc1a406e2b0 0x7fc1a4070770 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0000c00 tx=0x7fc1b00058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:30.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.772+0000 7fc1b97fa640 1 -- 172.21.15.53:0/218269170 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc1ac014030 con 0x7fc1bc102ab0 2024-09-19T06:42:30.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.959+0000 7fc1c2531640 1 -- 172.21.15.53:0/218269170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc1bc102e90 con 0x7fc1bc102ab0 2024-09-19T06:42:30.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.959+0000 7fc1b97fa640 1 -- 172.21.15.53:0/218269170 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v72) ==== 81+0+1001 (secure 0 0 0) 0x7fc1ac05f5c0 con 0x7fc1bc102ab0 2024-09-19T06:42:30.961 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.155:6833/390026217 2024-09-20T06:16:50.842891+0000 2024-09-19T06:42:30.961 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6801/3684090301 2024-09-20T06:11:28.721009+0000 2024-09-19T06:42:30.961 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6800/475342876 2024-09-20T06:09:58.271775+0000 2024-09-19T06:42:30.961 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6801/475342876 2024-09-20T06:09:58.271775+0000 2024-09-19T06:42:30.961 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.155:6832/390026217 2024-09-20T06:16:50.842891+0000 2024-09-19T06:42:30.962 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/3753705881 2024-09-20T06:11:28.721009+0000 2024-09-19T06:42:30.962 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/4216592427 2024-09-20T06:09:58.271775+0000 2024-09-19T06:42:30.962 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/1618433917 2024-09-20T06:10:30.209033+0000 2024-09-19T06:42:30.962 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/623598636 2024-09-20T06:09:58.271775+0000 2024-09-19T06:42:30.962 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6800/3684090301 2024-09-20T06:11:28.721009+0000 2024-09-19T06:42:30.962 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6800/681468213 2024-09-20T06:10:30.209033+0000 2024-09-19T06:42:30.962 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/597749918 2024-09-20T06:10:30.209033+0000 2024-09-19T06:42:30.962 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/21853845 2024-09-20T06:11:28.721009+0000 2024-09-19T06:42:30.962 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/3515303955 2024-09-20T06:09:58.271775+0000 2024-09-19T06:42:30.962 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6801/681468213 2024-09-20T06:10:30.209033+0000 2024-09-19T06:42:30.962 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/1729980812 2024-09-20T06:11:28.721009+0000 2024-09-19T06:42:30.962 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/372276981 2024-09-20T06:10:30.209033+0000 2024-09-19T06:42:30.963 INFO:teuthology.orchestra.run.smithi053.stderr:listed 17 entries 2024-09-19T06:42:30.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.961+0000 7fc1c2531640 1 -- 172.21.15.53:0/218269170 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fc1a406e2b0 msgr2=0x7fc1a4070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:30.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.961+0000 7fc1c2531640 1 --2- 172.21.15.53:0/218269170 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fc1a406e2b0 0x7fc1a4070770 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0000c00 tx=0x7fc1b00058d0 comp rx=0 tx=0).stop 2024-09-19T06:42:30.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.961+0000 7fc1c2531640 1 -- 172.21.15.53:0/218269170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1bc102ab0 msgr2=0x7fc1bc100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:30.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.961+0000 7fc1c2531640 1 --2- 172.21.15.53:0/218269170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1bc102ab0 0x7fc1bc100710 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac00c810 tx=0x7fc1ac00ccd0 comp rx=0 tx=0).stop 2024-09-19T06:42:30.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.961+0000 7fc1c2531640 1 -- 172.21.15.53:0/218269170 shutdown_connections 2024-09-19T06:42:30.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.961+0000 7fc1c2531640 1 --2- 172.21.15.53:0/218269170 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7fc1a406e2b0 0x7fc1a4070770 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:30.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.961+0000 7fc1c2531640 1 --2- 172.21.15.53:0/218269170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1bc1033d0 0x7fc1bc100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:30.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.961+0000 7fc1c2531640 1 --2- 172.21.15.53:0/218269170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1bc102ab0 0x7fc1bc100710 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:30.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.961+0000 7fc1c2531640 1 -- 172.21.15.53:0/218269170 >> 172.21.15.53:0/218269170 conn(0x7fc1bc0fc550 msgr2=0x7fc1bc103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:30.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.961+0000 7fc1c2531640 1 -- 172.21.15.53:0/218269170 shutdown_connections 2024-09-19T06:42:30.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:30.961+0000 7fc1c2531640 1 -- 172.21.15.53:0/218269170 wait complete. 2024-09-19T06:42:31.311 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:31 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/218269170' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T06:42:31.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:31 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/218269170' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T06:42:31.746 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-19T06:42:31.746 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T06:42:31.767 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-19T06:42:32.033 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/mon.smithi053/config 2024-09-19T06:42:32.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:32 smithi053 ceph-mon[27968]: pgmap v991: 129 pgs: 129 active+clean; 218 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 293 KiB/s wr, 56 op/s 2024-09-19T06:42:32.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:32 smithi155 ceph-mon[35292]: pgmap v991: 129 pgs: 129 active+clean; 218 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 293 KiB/s wr, 56 op/s 2024-09-19T06:42:32.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.477+0000 7f1b4b497640 1 -- 172.21.15.53:0/905616986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b44100800 msgr2=0x7f1b44100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:32.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.477+0000 7f1b4b497640 1 --2- 172.21.15.53:0/905616986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b44100800 0x7f1b44100c60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009880 tx=0x7f1b3402a4a0 comp rx=0 tx=0).stop 2024-09-19T06:42:32.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.477+0000 7f1b4b497640 1 -- 172.21.15.53:0/905616986 shutdown_connections 2024-09-19T06:42:32.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.477+0000 7f1b4b497640 1 --2- 172.21.15.53:0/905616986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b44100800 0x7f1b44100c60 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:32.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.477+0000 7f1b4b497640 1 --2- 172.21.15.53:0/905616986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b44106800 0x7f1b44106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:32.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.477+0000 7f1b4b497640 1 -- 172.21.15.53:0/905616986 >> 172.21.15.53:0/905616986 conn(0x7f1b440fc520 msgr2=0x7f1b440fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:32.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.477+0000 7f1b4b497640 1 -- 172.21.15.53:0/905616986 shutdown_connections 2024-09-19T06:42:32.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.477+0000 7f1b4b497640 1 -- 172.21.15.53:0/905616986 wait complete. 2024-09-19T06:42:32.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.478+0000 7f1b4b497640 1 Processor -- start 2024-09-19T06:42:32.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.478+0000 7f1b4b497640 1 -- start start 2024-09-19T06:42:32.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.479+0000 7f1b4b497640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b44100800 0x7f1b44198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:32.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.479+0000 7f1b4b497640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b44106800 0x7f1b44198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:32.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.479+0000 7f1b4b497640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4410dca0 con 0x7f1b44100800 2024-09-19T06:42:32.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.479+0000 7f1b4b497640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4410db20 con 0x7f1b44106800 2024-09-19T06:42:32.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.479+0000 7f1b49233640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b44100800 0x7f1b44198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:32.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.479+0000 7f1b49233640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b44100800 0x7f1b44198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56128/0 (socket says 172.21.15.53:56128) 2024-09-19T06:42:32.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.479+0000 7f1b49233640 1 -- 172.21.15.53:0/2582537298 learned_addr learned my addr 172.21.15.53:0/2582537298 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-19T06:42:32.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.479+0000 7f1b49233640 1 -- 172.21.15.53:0/2582537298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b44106800 msgr2=0x7f1b44198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:32.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.479+0000 7f1b49233640 1 --2- 172.21.15.53:0/2582537298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b44106800 0x7f1b44198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:32.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.479+0000 7f1b49233640 1 -- 172.21.15.53:0/2582537298 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b44199310 con 0x7f1b44100800 2024-09-19T06:42:32.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.479+0000 7f1b49233640 1 --2- 172.21.15.53:0/2582537298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b44100800 0x7f1b44198860 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000b420 tx=0x7f1b4000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:32.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.479+0000 7f1b3a7fc640 1 -- 172.21.15.53:0/2582537298 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1b40011630 con 0x7f1b44100800 2024-09-19T06:42:32.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.479+0000 7f1b3a7fc640 1 -- 172.21.15.53:0/2582537298 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b40016070 con 0x7f1b44100800 2024-09-19T06:42:32.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.479+0000 7f1b4b497640 1 -- 172.21.15.53:0/2582537298 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b44199600 con 0x7f1b44100800 2024-09-19T06:42:32.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.479+0000 7f1b3a7fc640 1 -- 172.21.15.53:0/2582537298 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1b40010c90 con 0x7f1b44100800 2024-09-19T06:42:32.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.479+0000 7f1b4b497640 1 -- 172.21.15.53:0/2582537298 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b441a1c00 con 0x7f1b44100800 2024-09-19T06:42:32.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.481+0000 7f1b4b497640 1 -- 172.21.15.53:0/2582537298 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b44101f40 con 0x7f1b44100800 2024-09-19T06:42:32.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.481+0000 7f1b3a7fc640 1 -- 172.21.15.53:0/2582537298 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92551+0+0 (secure 0 0 0) 0x7f1b400117d0 con 0x7f1b44100800 2024-09-19T06:42:32.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.481+0000 7f1b3a7fc640 1 --2- 172.21.15.53:0/2582537298 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f1b3006e310 0x7f1b300707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:32.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.482+0000 7f1b3a7fc640 1 -- 172.21.15.53:0/2582537298 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7340+0+0 (secure 0 0 0) 0x7f1b4008e120 con 0x7f1b44100800 2024-09-19T06:42:32.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.482+0000 7f1b48a32640 1 --2- 172.21.15.53:0/2582537298 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f1b3006e310 0x7f1b300707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:32.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.482+0000 7f1b48a32640 1 --2- 172.21.15.53:0/2582537298 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f1b3006e310 0x7f1b300707d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1b34004f10 tx=0x7f1b34000bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:32.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.486+0000 7f1b3a7fc640 1 -- 172.21.15.53:0/2582537298 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1b4005b320 con 0x7f1b44100800 2024-09-19T06:42:32.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.674+0000 7f1b4b497640 1 -- 172.21.15.53:0/2582537298 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1b44100ce0 con 0x7f1b44100800 2024-09-19T06:42:32.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.675+0000 7f1b3a7fc640 1 -- 172.21.15.53:0/2582537298 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v72) ==== 81+0+1001 (secure 0 0 0) 0x7f1b4005efd0 con 0x7f1b44100800 2024-09-19T06:42:32.677 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.155:6833/390026217 2024-09-20T06:16:50.842891+0000 2024-09-19T06:42:32.677 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6801/3684090301 2024-09-20T06:11:28.721009+0000 2024-09-19T06:42:32.677 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6800/475342876 2024-09-20T06:09:58.271775+0000 2024-09-19T06:42:32.677 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6801/475342876 2024-09-20T06:09:58.271775+0000 2024-09-19T06:42:32.677 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.155:6832/390026217 2024-09-20T06:16:50.842891+0000 2024-09-19T06:42:32.677 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/3753705881 2024-09-20T06:11:28.721009+0000 2024-09-19T06:42:32.677 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/4216592427 2024-09-20T06:09:58.271775+0000 2024-09-19T06:42:32.677 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/1618433917 2024-09-20T06:10:30.209033+0000 2024-09-19T06:42:32.677 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/623598636 2024-09-20T06:09:58.271775+0000 2024-09-19T06:42:32.678 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6800/3684090301 2024-09-20T06:11:28.721009+0000 2024-09-19T06:42:32.678 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6800/681468213 2024-09-20T06:10:30.209033+0000 2024-09-19T06:42:32.678 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/597749918 2024-09-20T06:10:30.209033+0000 2024-09-19T06:42:32.678 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/21853845 2024-09-20T06:11:28.721009+0000 2024-09-19T06:42:32.678 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/3515303955 2024-09-20T06:09:58.271775+0000 2024-09-19T06:42:32.678 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6801/681468213 2024-09-20T06:10:30.209033+0000 2024-09-19T06:42:32.678 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/1729980812 2024-09-20T06:11:28.721009+0000 2024-09-19T06:42:32.678 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/372276981 2024-09-20T06:10:30.209033+0000 2024-09-19T06:42:32.679 INFO:teuthology.orchestra.run.smithi053.stderr:listed 17 entries 2024-09-19T06:42:32.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.677+0000 7f1b4b497640 1 -- 172.21.15.53:0/2582537298 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f1b3006e310 msgr2=0x7f1b300707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:32.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.677+0000 7f1b4b497640 1 --2- 172.21.15.53:0/2582537298 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f1b3006e310 0x7f1b300707d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1b34004f10 tx=0x7f1b34000bc0 comp rx=0 tx=0).stop 2024-09-19T06:42:32.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.677+0000 7f1b4b497640 1 -- 172.21.15.53:0/2582537298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b44100800 msgr2=0x7f1b44198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:32.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.677+0000 7f1b4b497640 1 --2- 172.21.15.53:0/2582537298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b44100800 0x7f1b44198860 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000b420 tx=0x7f1b4000b8e0 comp rx=0 tx=0).stop 2024-09-19T06:42:32.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.677+0000 7f1b4b497640 1 -- 172.21.15.53:0/2582537298 shutdown_connections 2024-09-19T06:42:32.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.677+0000 7f1b4b497640 1 --2- 172.21.15.53:0/2582537298 >> [v2:172.21.15.53:6800/3884355752,v1:172.21.15.53:6801/3884355752] conn(0x7f1b3006e310 0x7f1b300707d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:32.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.677+0000 7f1b4b497640 1 --2- 172.21.15.53:0/2582537298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b44106800 0x7f1b44198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:32.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.677+0000 7f1b4b497640 1 --2- 172.21.15.53:0/2582537298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b44100800 0x7f1b44198860 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:32.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.677+0000 7f1b4b497640 1 -- 172.21.15.53:0/2582537298 >> 172.21.15.53:0/2582537298 conn(0x7f1b440fc520 msgr2=0x7f1b440fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:32.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.677+0000 7f1b4b497640 1 -- 172.21.15.53:0/2582537298 shutdown_connections 2024-09-19T06:42:32.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-19T06:42:32.677+0000 7f1b4b497640 1 -- 172.21.15.53:0/2582537298 wait complete. 2024-09-19T06:42:33.355 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:33 smithi053 ceph-mon[27968]: from='client.? 172.21.15.53:0/2582537298' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T06:42:33.396 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi155.front.sepia.ceph.com... 2024-09-19T06:42:33.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:42:33.397 DEBUG:teuthology.orchestra.run.smithi155:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-19T06:42:33.425 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:33 smithi155 ceph-mon[35292]: from='client.? 172.21.15.53:0/2582537298' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T06:42:33.439 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[64780]: fuse finished with error 0 and tester_r 0 2024-09-19T06:42:33.440 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T06:42:34.354 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:34 smithi053 ceph-mon[27968]: pgmap v992: 129 pgs: 129 active+clean; 218 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 19 KiB/s wr, 35 op/s 2024-09-19T06:42:34.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:34 smithi155 ceph-mon[35292]: pgmap v992: 129 pgs: 129 active+clean; 218 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 19 KiB/s wr, 35 op/s 2024-09-19T06:42:36.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:36 smithi155 ceph-mon[35292]: pgmap v993: 129 pgs: 129 active+clean; 218 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 85 B/s rd, 853 B/s wr, 1 op/s 2024-09-19T06:42:36.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:36 smithi053 ceph-mon[27968]: pgmap v993: 129 pgs: 129 active+clean; 218 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 85 B/s rd, 853 B/s wr, 1 op/s 2024-09-19T06:42:38.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:38 smithi155 ceph-mon[35292]: pgmap v994: 129 pgs: 129 active+clean; 218 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 85 B/s rd, 853 B/s wr, 1 op/s 2024-09-19T06:42:38.604 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:38 smithi053 ceph-mon[27968]: pgmap v994: 129 pgs: 129 active+clean; 218 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 85 B/s rd, 853 B/s wr, 1 op/s 2024-09-19T06:42:39.541 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2024-09-19T06:42:39.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:42:39.541 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T06:42:39.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:42:39.580 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2024-09-19T06:42:39.639 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-19T06:42:39.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:42:39.639 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T06:42:39.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:42:39.711 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link delete ceph-brx 2024-09-19T06:42:39.818 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-19T06:42:39.827 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-19T06:42:39.828 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-19T06:42:39.828 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T06:42:39.860 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20812 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/d2f9f09b1f48a651ccdd968737ef3a081e1766c8413dec44faa7542d15e6e4ec/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/V5YLV3QGME5KAMWDJXQPTYMIDJ:/var/lib/containers/storage/overlay/l/3S62E2NAQHCBF5SK4RPYPT5YYA,upperdir=/var/lib/containers/storage/overlay/d2f9f09b1f48a651ccdd968737ef3a081e1766c8413dec44faa7542d15e6e4ec/diff,workdir=/var/lib/containers/storage/overlay/d2f9f09b1f48a651ccdd968737ef3a081e1766c8413dec44faa7542d15e6e4ec/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/fdcc97a79881c7e5268b66a1628ca15dd0de697dae72455cc4a72b00338d30d5/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/V5YLV3QGME5KAMWDJXQPTYMIDJ:/var/lib/containers/storage/overlay/l/3S62E2NAQHCBF5SK4RPYPT5YYA,upperdir=/var/lib/containers/storage/overlay/fdcc97a79881c7e5268b66a1628ca15dd0de697dae72455cc4a72b00338d30d5/diff,workdir=/var/lib/containers/storage/overlay/fdcc97a79881c7e5268b66a1628ca15dd0de697dae72455cc4a72b00338d30d5/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/74c1fecbba935daf61b8fc346f42ea5fa869cec3eb263512c4e89bf02b607e56/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/V5YLV3QGME5KAMWDJXQPTYMIDJ:/var/lib/containers/storage/overlay/l/3S62E2NAQHCBF5SK4RPYPT5YYA,upperdir=/var/lib/containers/storage/overlay/74c1fecbba935daf61b8fc346f42ea5fa869cec3eb263512c4e89bf02b607e56/diff,workdir=/var/lib/containers/storage/overlay/74c1fecbba935daf61b8fc346f42ea5fa869cec3eb263512c4e89bf02b607e56/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/ae7613604d9abd182defb6d88563f1851a037934a3f97529b334b45bb53c7c3c/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/V5YLV3QGME5KAMWDJXQPTYMIDJ:/var/lib/containers/storage/overlay/l/3S62E2NAQHCBF5SK4RPYPT5YYA,upperdir=/var/lib/containers/storage/overlay/ae7613604d9abd182defb6d88563f1851a037934a3f97529b334b45bb53c7c3c/diff,workdir=/var/lib/containers/storage/overlay/ae7613604d9abd182defb6d88563f1851a037934a3f97529b334b45bb53c7c3c/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a816658b4a86d51683dc3724f9e5fc32de7456b6c93869a6ee7d9e789ba8224b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/K6DLWEPBCO56LTNOJCTOYM4SRA:/var/lib/containers/storage/overlay/l/L7GCFA52RR2UXMS25YQ3GNDG7D:/var/lib/containers/storage/overlay/l/FVR7RCUXPM7TW5SNEIRJVU2D4R,upperdir=/var/lib/containers/storage/overlay/a816658b4a86d51683dc3724f9e5fc32de7456b6c93869a6ee7d9e789ba8224b/diff,workdir=/var/lib/containers/storage/overlay/a816658b4a86d51683dc3724f9e5fc32de7456b6c93869a6ee7d9e789ba8224b/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/5a52dededbec74be706daf7110a780fb3cd39ea3b4ef7f266ca828dd43f98bcf/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/56VMZ4PBXPYV7KVBSLINIQCR5P:/var/lib/containers/storage/overlay/l/YDH2FE7B3FSTPYBBFCZEBHEBOC:/var/lib/containers/storage/overlay/l/NXDLJIIVND4ZV2WO3L4AE3H4WG:/var/lib/containers/storage/overlay/l/KSHUURRCTEZSTCJO5EA7J3GJVO:/var/lib/containers/storage/overlay/l/2D4LK4D3BENKSM33EATYJHNFWH:/var/lib/containers/storage/overlay/l/IJRNS6KZLNWY2OFOWSI47IYSBG,upperdir=/var/lib/containers/storage/overlay/5a52dededbec74be706daf7110a780fb3cd39ea3b4ef7f266ca828dd43f98bcf/diff,workdir=/var/lib/containers/storage/overlay/5a52dededbec74be706daf7110a780fb3cd39ea3b4ef7f266ca828dd43f98bcf/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a121757f5050c2845314fcb409ace16b64531733bc578520f5ca788fd7e32635/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/M5CUBYSJRN65FDEHARJSVCO3YQ:/var/lib/containers/storage/overlay/l/7SYLU4MQAWN46X5Y7CWXZGU5GG:/var/lib/containers/storage/overlay/l/7NVSNXMBZXFE7YUQNQ4XVMNDUT:/var/lib/containers/storage/overlay/l/MGB5DFHDZTWGOLUDI2BBYZDQID:/var/lib/containers/storage/overlay/l/EGVDN6U2LIQ72Y2IRDSHI5CSVH:/var/lib/containers/storage/overlay/l/ORQIHA3D3FNRIPMAC5DNBY2YR4:/var/lib/containers/storage/overlay/l/QUHBTLZ5YEEOGME7GIV4QK6NMW:/var/lib/containers/storage/overlay/l/JPJ7YGSMGLP4WEFB57WQNI4WZQ:/var/lib/containers/storage/overlay/l/SS2YGWISENG46EL5NGLXX5GOUM,upperdir=/var/lib/containers/storage/overlay/a121757f5050c2845314fcb409ace16b64531733bc578520f5ca788fd7e32635/diff,workdir=/var/lib/containers/storage/overlay/a121757f5050c2845314fcb409ace16b64531733bc578520f5ca788fd7e32635/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/ea991316be66cf2df0b42db33f3cb5faae616f6f70accdf06e5caa03547fd221/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/AERQS5BKQHUA5RYSZ5BR53ZS7B:/var/lib/containers/storage/overlay/l/K5AJ34XQJAB6IX4LH5CIFQBT6C:/var/lib/containers/storage/overlay/l/IIRL4NATV7GGLCVPGWVP7DHLFM:/var/lib/containers/storage/overlay/l/MCTCKTOLH2H5LBOUKHRJQLY635:/var/lib/containers/storage/overlay/l/IZJ27XD47PIJR4TT6X7PXYLY55:/var/lib/containers/storage/overlay/l/Y5SF37XGQ3W5EZS6LGDTS4XEJN:/var/lib/containers/storage/overlay/l/52L6Q3K7DYBAFJOEND5L4IRMFI:/var/lib/containers/storage/overlay/l/TXTNEYFGR32CPJWZWPPS55OGCI:/var/lib/containers/storage/overlay/l/LIPMKPYSFHKJMIFCWAVB3WIEY3:/var/lib/containers/storage/overlay/l/VZGZ3RWUOZB6O4VRW47HD67FBK:/var/lib/containers/storage/overlay/l/YD4VNW3M4U52YKMWJG672OR3X6:/var/lib/containers/storage/overlay/l/6OPSBULWNCL7ECKPTNDLD74LFV,upperdir=/var/lib/containers/storage/overlay/ea991316be66cf2df0b42db33f3cb5faae616f6f70accdf06e5caa03547fd221/diff,workdir=/var/lib/containers/storage/overlay/ea991316be66cf2df0b42db33f3cb5faae616f6f70accdf06e5caa03547fd221/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/e5ef75b955272730be5fc2ddd5e539cc3ea309e8910ca0fd6dffae8b9e783016/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/V5YLV3QGME5KAMWDJXQPTYMIDJ:/var/lib/containers/storage/overlay/l/3S62E2NAQHCBF5SK4RPYPT5YYA,upperdir=/var/lib/containers/storage/overlay/e5ef75b955272730be5fc2ddd5e539cc3ea309e8910ca0fd6dffae8b9e783016/diff,workdir=/var/lib/containers/storage/overlay/e5ef75b955272730be5fc2ddd5e539cc3ea309e8910ca0fd6dffae8b9e783016/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/c547cd5231027c601197986ecf6254ff69a15b572df09f9cf80bc870101fe024/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/V5YLV3QGME5KAMWDJXQPTYMIDJ:/var/lib/containers/storage/overlay/l/3S62E2NAQHCBF5SK4RPYPT5YYA,upperdir=/var/lib/containers/storage/overlay/c547cd5231027c601197986ecf6254ff69a15b572df09f9cf80bc870101fe024/diff,workdir=/var/lib/containers/storage/overlay/c547cd5231027c601197986ecf6254ff69a15b572df09f9cf80bc870101fe024/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/e41c9b603e693ec4f9f2141cd12ffb8516aa4bcf28f584457ce7dd1596aaa69c/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/V5YLV3QGME5KAMWDJXQPTYMIDJ:/var/lib/containers/storage/overlay/l/3S62E2NAQHCBF5SK4RPYPT5YYA,upperdir=/var/lib/containers/storage/overlay/e41c9b603e693ec4f9f2141cd12ffb8516aa4bcf28f584457ce7dd1596aaa69c/diff,workdir=/var/lib/containers/storage/overlay/e41c9b603e693ec4f9f2141cd12ffb8516aa4bcf28f584457ce7dd1596aaa69c/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/32a10f17d5ede08bf0ac8706e5874294f65c93d601490778b893cdfbbb501d96/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/V5YLV3QGME5KAMWDJXQPTYMIDJ:/var/lib/containers/storage/overlay/l/3S62E2NAQHCBF5SK4RPYPT5YYA,upperdir=/var/lib/containers/storage/overlay/32a10f17d5ede08bf0ac8706e5874294f65c93d601490778b893cdfbbb501d96/diff,workdir=/var/lib/containers/storage/overlay/32a10f17d5ede08bf0ac8706e5874294f65c93d601490778b893cdfbbb501d96/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/3a08cd75f0291bcbfbbb041f18eb2ce1fee7e6ee52e2c401dfa92e4fe0c6e14e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/V5YLV3QGME5KAMWDJXQPTYMIDJ:/var/lib/containers/storage/overlay/l/3S62E2NAQHCBF5SK4RPYPT5YYA,upperdir=/var/lib/containers/storage/overlay/3a08cd75f0291bcbfbbb041f18eb2ce1fee7e6ee52e2c401dfa92e4fe0c6e14e/diff,workdir=/var/lib/containers/storage/overlay/3a08cd75f0291bcbfbbb041f18eb2ce1fee7e6ee52e2c401dfa92e4fe0c6e14e/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f29ea1ea7e7e57fc673cfd38b79254ec0fce094be194ace9fe778619bf886b1f/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/V5YLV3QGME5KAMWDJXQPTYMIDJ:/var/lib/containers/storage/overlay/l/3S62E2NAQHCBF5SK4RPYPT5YYA,upperdir=/var/lib/containers/storage/overlay/f29ea1ea7e7e57fc673cfd38b79254ec0fce094be194ace9fe778619bf886b1f/diff,workdir=/var/lib/containers/storage/overlay/f29ea1ea7e7e57fc673cfd38b79254ec0fce094be194ace9fe778619bf886b1f/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-09-19T06:42:39.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:42:39.860 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-09-19T06:42:39.920 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T06:42:39.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:42:39.921 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T06:42:39.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:42:39.996 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link delete ceph-brx 2024-09-19T06:42:40.089 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-19T06:42:40.100 INFO:tasks.cephadm:Teardown begin 2024-09-19T06:42:40.100 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T06:42:40.137 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T06:42:40.169 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-19T06:42:40.169 DEBUG:teuthology.orchestra.run.smithi053:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T06:42:40.194 DEBUG:teuthology.orchestra.run.smithi155:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T06:42:40.227 INFO:tasks.cephadm:Stopping all daemons... 2024-09-19T06:42:40.227 INFO:tasks.cephadm.mon.smithi053:Stopping mon.smithi053... 2024-09-19T06:42:40.227 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-9f2876d0-764d-11ef-bceb-c7b262605968@mon.smithi053 2024-09-19T06:42:40.257 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:40 smithi053 ceph-mon[27968]: pgmap v995: 129 pgs: 129 active+clean; 218 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 85 B/s rd, 853 B/s wr, 0 op/s 2024-09-19T06:42:40.546 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:40 smithi053 systemd[1]: Stopping Ceph mon.smithi053 for 9f2876d0-764d-11ef-bceb-c7b262605968... 2024-09-19T06:42:40.732 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:40 smithi155 ceph-mon[35292]: pgmap v995: 129 pgs: 129 active+clean; 218 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 85 B/s rd, 853 B/s wr, 0 op/s 2024-09-19T06:42:40.854 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:40 smithi053 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-mon-smithi053[27964]: 2024-09-19T06:42:40.543+0000 7f0718de2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi053 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:42:40.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:40 smithi053 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-mon-smithi053[27964]: 2024-09-19T06:42:40.543+0000 7f0718de2640 -1 mon.smithi053@0(leader) e2 *** Got Signal Terminated *** 2024-09-19T06:42:40.855 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:40 smithi053 podman[85358]: 2024-09-19 06:42:40.646557927 +0000 UTC m=+0.209678886 container died 3a3c39f8318dc4298a0aeedccb7c17526d4a93ddf388f71f17f5a78fcf5e7433 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-mon-smithi053, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-19T06:42:41.329 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:41 smithi053 podman[85358]: 2024-09-19 06:42:41.069244107 +0000 UTC m=+0.632365074 container cleanup 3a3c39f8318dc4298a0aeedccb7c17526d4a93ddf388f71f17f5a78fcf5e7433 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-mon-smithi053, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4) 2024-09-19T06:42:41.329 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:41 smithi053 bash[85358]: ceph-9f2876d0-764d-11ef-bceb-c7b262605968-mon-smithi053 2024-09-19T06:42:41.572 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f2876d0-764d-11ef-bceb-c7b262605968@mon.smithi053.service' 2024-09-19T06:42:41.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:41 smithi053 podman[85370]: 2024-09-19 06:42:41.328817849 +0000 UTC m=+0.684620079 container remove 3a3c39f8318dc4298a0aeedccb7c17526d4a93ddf388f71f17f5a78fcf5e7433 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-mon-smithi053, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-19T06:42:41.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:41 smithi053 systemd[1]: ceph-9f2876d0-764d-11ef-bceb-c7b262605968@mon.smithi053.service: Deactivated successfully. 2024-09-19T06:42:41.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:41 smithi053 systemd[1]: Stopped Ceph mon.smithi053 for 9f2876d0-764d-11ef-bceb-c7b262605968. 2024-09-19T06:42:41.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 19 06:42:41 smithi053 systemd[1]: ceph-9f2876d0-764d-11ef-bceb-c7b262605968@mon.smithi053.service: Consumed 21.690s CPU time. 2024-09-19T06:42:41.657 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:42:41.658 INFO:tasks.cephadm.mon.smithi053:Stopped mon.smithi053 2024-09-19T06:42:41.658 INFO:tasks.cephadm.mon.smithi155:Stopping mon.smithi155... 2024-09-19T06:42:41.658 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-9f2876d0-764d-11ef-bceb-c7b262605968@mon.smithi155 2024-09-19T06:42:41.982 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:41 smithi155 systemd[1]: Stopping Ceph mon.smithi155 for 9f2876d0-764d-11ef-bceb-c7b262605968... 2024-09-19T06:42:42.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:42 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-mon-smithi155[35288]: 2024-09-19T06:42:42.041+0000 7fc4bf55b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi155 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:42:42.482 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:42 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-mon-smithi155[35288]: 2024-09-19T06:42:42.041+0000 7fc4bf55b640 -1 mon.smithi155@1(peon) e2 *** Got Signal Terminated *** 2024-09-19T06:42:42.483 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:42 smithi155 podman[71934]: 2024-09-19 06:42:42.205250292 +0000 UTC m=+0.300857530 container died 043f44f8feb0f1dd9f903a987dd746c2ffce335378a7bf6f1f27939b6d1bc8eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-mon-smithi155, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:42:43.103 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:42 smithi155 podman[71934]: 2024-09-19 06:42:42.842612541 +0000 UTC m=+0.938219767 container cleanup 043f44f8feb0f1dd9f903a987dd746c2ffce335378a7bf6f1f27939b6d1bc8eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-mon-smithi155, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-19T06:42:43.103 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:42 smithi155 bash[71934]: ceph-9f2876d0-764d-11ef-bceb-c7b262605968-mon-smithi155 2024-09-19T06:42:43.358 INFO:journalctl@ceph.mon.smithi155.smithi155.stdout:Sep 19 06:42:43 smithi155 podman[71948]: 2024-09-19 06:42:43.102992685 +0000 UTC m=+0.877439705 container remove 043f44f8feb0f1dd9f903a987dd746c2ffce335378a7bf6f1f27939b6d1bc8eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-mon-smithi155, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:42:43.377 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f2876d0-764d-11ef-bceb-c7b262605968@mon.smithi155.service' 2024-09-19T06:42:43.426 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:42:43.426 INFO:tasks.cephadm.mon.smithi155:Stopped mon.smithi155 2024-09-19T06:42:43.427 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-19T06:42:43.427 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-9f2876d0-764d-11ef-bceb-c7b262605968@osd.0 2024-09-19T06:42:43.784 INFO:journalctl@ceph.osd.0.smithi053.stdout:Sep 19 06:42:43 smithi053 systemd[1]: Stopping Ceph osd.0 for 9f2876d0-764d-11ef-bceb-c7b262605968... 2024-09-19T06:42:44.105 INFO:journalctl@ceph.osd.0.smithi053.stdout:Sep 19 06:42:43 smithi053 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-0[46154]: 2024-09-19T06:42:43.781+0000 7f7078581640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:42:44.105 INFO:journalctl@ceph.osd.0.smithi053.stdout:Sep 19 06:42:43 smithi053 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-0[46154]: 2024-09-19T06:42:43.781+0000 7f7078581640 -1 osd.0 72 *** Got signal Terminated *** 2024-09-19T06:42:44.105 INFO:journalctl@ceph.osd.0.smithi053.stdout:Sep 19 06:42:43 smithi053 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-0[46154]: 2024-09-19T06:42:43.781+0000 7f7078581640 -1 osd.0 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:42:49.513 INFO:journalctl@ceph.osd.0.smithi053.stdout:Sep 19 06:42:49 smithi053 podman[85490]: 2024-09-19 06:42:49.207610269 +0000 UTC m=+5.574695869 container died e2d04a79b63737cca08ac3bdfeacaad53fe984c28a855ee15ba58c21f2349b51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:42:49.855 INFO:journalctl@ceph.osd.0.smithi053.stdout:Sep 19 06:42:49 smithi053 podman[85490]: 2024-09-19 06:42:49.607211042 +0000 UTC m=+5.974296641 container cleanup e2d04a79b63737cca08ac3bdfeacaad53fe984c28a855ee15ba58c21f2349b51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-0, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731) 2024-09-19T06:42:49.855 INFO:journalctl@ceph.osd.0.smithi053.stdout:Sep 19 06:42:49 smithi053 bash[85490]: ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-0 2024-09-19T06:42:50.162 INFO:journalctl@ceph.osd.0.smithi053.stdout:Sep 19 06:42:49 smithi053 podman[85508]: 2024-09-19 06:42:49.900591741 +0000 UTC m=+0.673877585 container remove e2d04a79b63737cca08ac3bdfeacaad53fe984c28a855ee15ba58c21f2349b51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:42:50.496 INFO:journalctl@ceph.osd.0.smithi053.stdout:Sep 19 06:42:50 smithi053 podman[85597]: 2024-09-19 06:42:50.126601861 +0000 UTC m=+0.016743830 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:42:50.497 INFO:journalctl@ceph.osd.0.smithi053.stdout:Sep 19 06:42:50 smithi053 podman[85597]: 2024-09-19 06:42:50.22641157 +0000 UTC m=+0.116553525 container create 178881f0a400b100d7ed9cea444db7643b3be37b6866bf55c4929acbcda46066 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, ceph=True, GIT_BRANCH=HEAD) 2024-09-19T06:42:50.497 INFO:journalctl@ceph.osd.0.smithi053.stdout:Sep 19 06:42:50 smithi053 podman[85597]: 2024-09-19 06:42:50.495701552 +0000 UTC m=+0.385843514 container init 178881f0a400b100d7ed9cea444db7643b3be37b6866bf55c4929acbcda46066 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-19T06:42:50.852 INFO:journalctl@ceph.osd.0.smithi053.stdout:Sep 19 06:42:50 smithi053 podman[85597]: 2024-09-19 06:42:50.499445496 +0000 UTC m=+0.389587455 container start 178881f0a400b100d7ed9cea444db7643b3be37b6866bf55c4929acbcda46066 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:42:50.852 INFO:journalctl@ceph.osd.0.smithi053.stdout:Sep 19 06:42:50 smithi053 podman[85597]: 2024-09-19 06:42:50.597996799 +0000 UTC m=+0.488138767 container attach 178881f0a400b100d7ed9cea444db7643b3be37b6866bf55c4929acbcda46066 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:42:51.104 INFO:journalctl@ceph.osd.0.smithi053.stdout:Sep 19 06:42:50 smithi053 podman[85597]: 2024-09-19 06:42:50.852195569 +0000 UTC m=+0.742337535 container died 178881f0a400b100d7ed9cea444db7643b3be37b6866bf55c4929acbcda46066 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-19T06:42:51.480 INFO:journalctl@ceph.osd.0.smithi053.stdout:Sep 19 06:42:51 smithi053 podman[85620]: 2024-09-19 06:42:51.476889187 +0000 UTC m=+0.618619499 container remove 178881f0a400b100d7ed9cea444db7643b3be37b6866bf55c4929acbcda46066 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-19T06:42:51.496 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f2876d0-764d-11ef-bceb-c7b262605968@osd.0.service' 2024-09-19T06:42:51.538 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:42:51.538 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-19T06:42:51.538 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-19T06:42:51.539 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-9f2876d0-764d-11ef-bceb-c7b262605968@osd.1 2024-09-19T06:42:51.855 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 19 06:42:51 smithi053 systemd[1]: Stopping Ceph osd.1 for 9f2876d0-764d-11ef-bceb-c7b262605968... 2024-09-19T06:42:52.355 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 19 06:42:51 smithi053 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-1[52663]: 2024-09-19T06:42:51.890+0000 7f9452216640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:42:52.355 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 19 06:42:51 smithi053 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-1[52663]: 2024-09-19T06:42:51.890+0000 7f9452216640 -1 osd.1 72 *** Got signal Terminated *** 2024-09-19T06:42:52.355 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 19 06:42:51 smithi053 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-1[52663]: 2024-09-19T06:42:51.890+0000 7f9452216640 -1 osd.1 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:42:57.716 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 19 06:42:57 smithi053 podman[85701]: 2024-09-19 06:42:57.425684471 +0000 UTC m=+5.644622660 container died 771e86d6a506424f603ba87e22fad13417fbcf71dc82cab7031708a39f6592b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-1, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4) 2024-09-19T06:42:58.082 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 19 06:42:57 smithi053 podman[85701]: 2024-09-19 06:42:57.805125534 +0000 UTC m=+6.024063714 container cleanup 771e86d6a506424f603ba87e22fad13417fbcf71dc82cab7031708a39f6592b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-1, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-19T06:42:58.082 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 19 06:42:57 smithi053 bash[85701]: ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-1 2024-09-19T06:42:58.354 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 19 06:42:58 smithi053 podman[85717]: 2024-09-19 06:42:58.081816371 +0000 UTC m=+0.646182377 container remove 771e86d6a506424f603ba87e22fad13417fbcf71dc82cab7031708a39f6592b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-1, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-19T06:42:58.703 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 19 06:42:58 smithi053 podman[85807]: 2024-09-19 06:42:58.316922828 +0000 UTC m=+0.016541016 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:42:58.703 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 19 06:42:58 smithi053 podman[85807]: 2024-09-19 06:42:58.416047485 +0000 UTC m=+0.115665674 container create 4c3350bdbec23652942bd192ef22892d179c1a4250673c60e18906b9e2328a9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-1-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-19T06:42:58.703 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 19 06:42:58 smithi053 podman[85807]: 2024-09-19 06:42:58.643688516 +0000 UTC m=+0.343306704 container init 4c3350bdbec23652942bd192ef22892d179c1a4250673c60e18906b9e2328a9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-19T06:42:58.704 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 19 06:42:58 smithi053 podman[85807]: 2024-09-19 06:42:58.648471681 +0000 UTC m=+0.348089874 container start 4c3350bdbec23652942bd192ef22892d179c1a4250673c60e18906b9e2328a9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True) 2024-09-19T06:42:58.704 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 19 06:42:58 smithi053 podman[85807]: 2024-09-19 06:42:58.70185268 +0000 UTC m=+0.401470870 container attach 4c3350bdbec23652942bd192ef22892d179c1a4250673c60e18906b9e2328a9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-19T06:42:59.104 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 19 06:42:58 smithi053 podman[85807]: 2024-09-19 06:42:58.845269267 +0000 UTC m=+0.544887454 container died 4c3350bdbec23652942bd192ef22892d179c1a4250673c60e18906b9e2328a9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-09-19T06:42:59.426 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f2876d0-764d-11ef-bceb-c7b262605968@osd.1.service' 2024-09-19T06:42:59.469 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:42:59.470 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-19T06:42:59.470 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-19T06:42:59.470 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-9f2876d0-764d-11ef-bceb-c7b262605968@osd.2 2024-09-19T06:42:59.827 INFO:journalctl@ceph.osd.2.smithi053.stdout:Sep 19 06:42:59 smithi053 systemd[1]: Stopping Ceph osd.2 for 9f2876d0-764d-11ef-bceb-c7b262605968... 2024-09-19T06:43:00.105 INFO:journalctl@ceph.osd.2.smithi053.stdout:Sep 19 06:42:59 smithi053 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-2[59344]: 2024-09-19T06:42:59.825+0000 7fde66269640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:43:00.105 INFO:journalctl@ceph.osd.2.smithi053.stdout:Sep 19 06:42:59 smithi053 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-2[59344]: 2024-09-19T06:42:59.825+0000 7fde66269640 -1 osd.2 72 *** Got signal Terminated *** 2024-09-19T06:43:00.105 INFO:journalctl@ceph.osd.2.smithi053.stdout:Sep 19 06:42:59 smithi053 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-2[59344]: 2024-09-19T06:42:59.825+0000 7fde66269640 -1 osd.2 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:43:05.486 INFO:journalctl@ceph.osd.2.smithi053.stdout:Sep 19 06:43:05 smithi053 podman[85909]: 2024-09-19 06:43:05.21328801 +0000 UTC m=+5.507471624 container died 18e322e6f311fde1ee6db89e2f91e71b0e63e02791f5cb82a59722a439622035 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:43:05.855 INFO:journalctl@ceph.osd.2.smithi053.stdout:Sep 19 06:43:05 smithi053 podman[85909]: 2024-09-19 06:43:05.559456681 +0000 UTC m=+5.853640281 container cleanup 18e322e6f311fde1ee6db89e2f91e71b0e63e02791f5cb82a59722a439622035 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-19T06:43:05.855 INFO:journalctl@ceph.osd.2.smithi053.stdout:Sep 19 06:43:05 smithi053 bash[85909]: ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-2 2024-09-19T06:43:06.354 INFO:journalctl@ceph.osd.2.smithi053.stdout:Sep 19 06:43:05 smithi053 podman[85938]: 2024-09-19 06:43:05.977779124 +0000 UTC m=+0.761858273 container remove 18e322e6f311fde1ee6db89e2f91e71b0e63e02791f5cb82a59722a439622035 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-19T06:43:06.707 INFO:journalctl@ceph.osd.2.smithi053.stdout:Sep 19 06:43:06 smithi053 podman[86026]: 2024-09-19 06:43:06.345583973 +0000 UTC m=+0.016708148 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:43:06.707 INFO:journalctl@ceph.osd.2.smithi053.stdout:Sep 19 06:43:06 smithi053 podman[86026]: 2024-09-19 06:43:06.453730567 +0000 UTC m=+0.124854728 container create 351b9cbbb4a977146f5de6b005f57ffa61744bad3311db891771187bcc49c726 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-19T06:43:06.707 INFO:journalctl@ceph.osd.2.smithi053.stdout:Sep 19 06:43:06 smithi053 podman[86026]: 2024-09-19 06:43:06.706616303 +0000 UTC m=+0.377740474 container init 351b9cbbb4a977146f5de6b005f57ffa61744bad3311db891771187bcc49c726 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:43:07.105 INFO:journalctl@ceph.osd.2.smithi053.stdout:Sep 19 06:43:06 smithi053 podman[86026]: 2024-09-19 06:43:06.710367785 +0000 UTC m=+0.381491951 container start 351b9cbbb4a977146f5de6b005f57ffa61744bad3311db891771187bcc49c726 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:43:07.105 INFO:journalctl@ceph.osd.2.smithi053.stdout:Sep 19 06:43:06 smithi053 podman[86026]: 2024-09-19 06:43:06.798332497 +0000 UTC m=+0.469456672 container attach 351b9cbbb4a977146f5de6b005f57ffa61744bad3311db891771187bcc49c726 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-19T06:43:07.105 INFO:journalctl@ceph.osd.2.smithi053.stdout:Sep 19 06:43:06 smithi053 podman[86026]: 2024-09-19 06:43:06.915450879 +0000 UTC m=+0.586575054 container died 351b9cbbb4a977146f5de6b005f57ffa61744bad3311db891771187bcc49c726 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-19T06:43:07.493 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f2876d0-764d-11ef-bceb-c7b262605968@osd.2.service' 2024-09-19T06:43:07.537 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:43:07.538 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-19T06:43:07.538 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-19T06:43:07.538 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-9f2876d0-764d-11ef-bceb-c7b262605968@osd.3 2024-09-19T06:43:07.855 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 19 06:43:07 smithi053 systemd[1]: Stopping Ceph osd.3 for 9f2876d0-764d-11ef-bceb-c7b262605968... 2024-09-19T06:43:08.355 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 19 06:43:07 smithi053 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-3[66229]: 2024-09-19T06:43:07.890+0000 7f674bd2b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:43:08.355 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 19 06:43:07 smithi053 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-3[66229]: 2024-09-19T06:43:07.890+0000 7f674bd2b640 -1 osd.3 72 *** Got signal Terminated *** 2024-09-19T06:43:08.355 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 19 06:43:07 smithi053 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-3[66229]: 2024-09-19T06:43:07.890+0000 7f674bd2b640 -1 osd.3 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:43:09.604 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 19 06:43:09 smithi053 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-3[66229]: 2024-09-19T06:43:09.332+0000 7f6744339640 -1 osd.3 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:46.423538+0000 front 2024-09-19T06:42:46.423727+0000 (oldest deadline 2024-09-19T06:43:09.323801+0000) 2024-09-19T06:43:10.605 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 19 06:43:10 smithi053 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-3[66229]: 2024-09-19T06:43:10.315+0000 7f6744339640 -1 osd.3 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:46.423538+0000 front 2024-09-19T06:42:46.423727+0000 (oldest deadline 2024-09-19T06:43:09.323801+0000) 2024-09-19T06:43:10.982 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:10 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:10.729+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.430806+0000 front 2024-09-19T06:42:48.430819+0000 (oldest deadline 2024-09-19T06:43:10.130557+0000) 2024-09-19T06:43:11.483 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:11 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:11.082+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:44.915444+0000 front 2024-09-19T06:42:44.915317+0000 (oldest deadline 2024-09-19T06:43:10.815165+0000) 2024-09-19T06:43:11.605 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 19 06:43:11 smithi053 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-3[66229]: 2024-09-19T06:43:11.268+0000 7f6744339640 -1 osd.3 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:46.423538+0000 front 2024-09-19T06:42:46.423727+0000 (oldest deadline 2024-09-19T06:43:09.323801+0000) 2024-09-19T06:43:11.982 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:11 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:11.713+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.430806+0000 front 2024-09-19T06:42:48.430819+0000 (oldest deadline 2024-09-19T06:43:10.130557+0000) 2024-09-19T06:43:12.482 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:12 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:12.037+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:44.915444+0000 front 2024-09-19T06:42:44.915317+0000 (oldest deadline 2024-09-19T06:43:10.815165+0000) 2024-09-19T06:43:12.605 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 19 06:43:12 smithi053 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-3[66229]: 2024-09-19T06:43:12.307+0000 7f6744339640 -1 osd.3 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:46.423538+0000 front 2024-09-19T06:42:46.423727+0000 (oldest deadline 2024-09-19T06:43:09.323801+0000) 2024-09-19T06:43:12.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:12 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:12.579+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:13.061 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:12 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:12.741+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.430806+0000 front 2024-09-19T06:42:48.430819+0000 (oldest deadline 2024-09-19T06:43:10.130557+0000) 2024-09-19T06:43:13.482 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:13 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:13.060+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:44.915444+0000 front 2024-09-19T06:42:44.915317+0000 (oldest deadline 2024-09-19T06:43:10.815165+0000) 2024-09-19T06:43:13.605 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 19 06:43:13 smithi053 podman[86130]: 2024-09-19 06:43:13.33153568 +0000 UTC m=+5.562626293 container died ba2d56fafe1e89d263210de2e4470054cc0c7ab174b5e844a8ff3948a5662152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-19T06:43:13.974 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 19 06:43:13 smithi053 podman[86130]: 2024-09-19 06:43:13.722029785 +0000 UTC m=+5.953120390 container cleanup ba2d56fafe1e89d263210de2e4470054cc0c7ab174b5e844a8ff3948a5662152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-3, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:43:13.974 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 19 06:43:13 smithi053 bash[86130]: ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-3 2024-09-19T06:43:13.982 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:13 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:13.758+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.430806+0000 front 2024-09-19T06:42:48.430819+0000 (oldest deadline 2024-09-19T06:43:10.130557+0000) 2024-09-19T06:43:13.982 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:13 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:13.565+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:14.308 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 19 06:43:13 smithi053 podman[86146]: 2024-09-19 06:43:13.973612124 +0000 UTC m=+0.643100018 container remove ba2d56fafe1e89d263210de2e4470054cc0c7ab174b5e844a8ff3948a5662152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-19T06:43:14.482 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:14 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:14.072+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:44.915444+0000 front 2024-09-19T06:42:44.915317+0000 (oldest deadline 2024-09-19T06:43:10.815165+0000) 2024-09-19T06:43:14.577 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 19 06:43:14 smithi053 podman[86232]: 2024-09-19 06:43:14.210440036 +0000 UTC m=+0.016716479 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:43:14.577 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 19 06:43:14 smithi053 podman[86232]: 2024-09-19 06:43:14.332699991 +0000 UTC m=+0.138976461 container create 310d2abc81126bcb8d1a5fa9a328db4120b40f1073b5bfbfd4fda6272d00e078 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, ceph=True) 2024-09-19T06:43:14.577 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 19 06:43:14 smithi053 podman[86232]: 2024-09-19 06:43:14.576992564 +0000 UTC m=+0.383269000 container init 310d2abc81126bcb8d1a5fa9a328db4120b40f1073b5bfbfd4fda6272d00e078 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-19T06:43:14.855 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 19 06:43:14 smithi053 podman[86232]: 2024-09-19 06:43:14.580544315 +0000 UTC m=+0.386820748 container start 310d2abc81126bcb8d1a5fa9a328db4120b40f1073b5bfbfd4fda6272d00e078 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-3-deactivate, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-19T06:43:14.855 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 19 06:43:14 smithi053 podman[86232]: 2024-09-19 06:43:14.685373104 +0000 UTC m=+0.491649532 container attach 310d2abc81126bcb8d1a5fa9a328db4120b40f1073b5bfbfd4fda6272d00e078 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-19T06:43:14.855 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 19 06:43:14 smithi053 podman[86232]: 2024-09-19 06:43:14.772141657 +0000 UTC m=+0.578418101 container died 310d2abc81126bcb8d1a5fa9a328db4120b40f1073b5bfbfd4fda6272d00e078 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-19T06:43:14.982 INFO:journalctl@ceph.osd.4.smithi155.stdout:Sep 19 06:43:14 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-4[40915]: 2024-09-19T06:43:14.672+0000 7f7d514d2640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.418462+0000 front 2024-09-19T06:42:48.418301+0000 (oldest deadline 2024-09-19T06:43:13.718207+0000) 2024-09-19T06:43:14.983 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:14 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:14.761+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.430806+0000 front 2024-09-19T06:42:48.430819+0000 (oldest deadline 2024-09-19T06:43:10.130557+0000) 2024-09-19T06:43:14.983 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:14 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:14.608+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:15.397 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f2876d0-764d-11ef-bceb-c7b262605968@osd.3.service' 2024-09-19T06:43:15.441 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:43:15.442 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-19T06:43:15.442 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-19T06:43:15.442 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-9f2876d0-764d-11ef-bceb-c7b262605968@osd.4 2024-09-19T06:43:15.472 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:15 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:15.028+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:44.915444+0000 front 2024-09-19T06:42:44.915317+0000 (oldest deadline 2024-09-19T06:43:10.815165+0000) 2024-09-19T06:43:15.733 INFO:journalctl@ceph.osd.4.smithi155.stdout:Sep 19 06:43:15 smithi155 systemd[1]: Stopping Ceph osd.4 for 9f2876d0-764d-11ef-bceb-c7b262605968... 2024-09-19T06:43:15.733 INFO:journalctl@ceph.osd.4.smithi155.stdout:Sep 19 06:43:15 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-4[40915]: 2024-09-19T06:43:15.652+0000 7f7d514d2640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.418462+0000 front 2024-09-19T06:42:48.418301+0000 (oldest deadline 2024-09-19T06:43:13.718207+0000) 2024-09-19T06:43:15.733 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:15 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:15.586+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:16.061 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:15 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:15.754+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.430806+0000 front 2024-09-19T06:42:48.430819+0000 (oldest deadline 2024-09-19T06:43:10.130557+0000) 2024-09-19T06:43:16.061 INFO:journalctl@ceph.osd.4.smithi155.stdout:Sep 19 06:43:15 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-4[40915]: 2024-09-19T06:43:15.780+0000 7f7d58ec4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:43:16.062 INFO:journalctl@ceph.osd.4.smithi155.stdout:Sep 19 06:43:15 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-4[40915]: 2024-09-19T06:43:15.780+0000 7f7d58ec4640 -1 osd.4 72 *** Got signal Terminated *** 2024-09-19T06:43:16.062 INFO:journalctl@ceph.osd.4.smithi155.stdout:Sep 19 06:43:15 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-4[40915]: 2024-09-19T06:43:15.780+0000 7f7d58ec4640 -1 osd.4 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:43:16.482 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:16 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:16.060+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:44.915444+0000 front 2024-09-19T06:42:44.915317+0000 (oldest deadline 2024-09-19T06:43:10.815165+0000) 2024-09-19T06:43:16.982 INFO:journalctl@ceph.osd.4.smithi155.stdout:Sep 19 06:43:16 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-4[40915]: 2024-09-19T06:43:16.679+0000 7f7d514d2640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.418462+0000 front 2024-09-19T06:42:48.418301+0000 (oldest deadline 2024-09-19T06:43:13.718207+0000) 2024-09-19T06:43:16.983 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:16 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:16.749+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.430806+0000 front 2024-09-19T06:42:48.430819+0000 (oldest deadline 2024-09-19T06:43:10.130557+0000) 2024-09-19T06:43:16.983 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:16 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:16.540+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:17.482 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:17 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:17.025+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:44.915444+0000 front 2024-09-19T06:42:44.915317+0000 (oldest deadline 2024-09-19T06:43:10.815165+0000) 2024-09-19T06:43:17.787 INFO:journalctl@ceph.osd.4.smithi155.stdout:Sep 19 06:43:17 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-4[40915]: 2024-09-19T06:43:17.663+0000 7f7d514d2640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.418462+0000 front 2024-09-19T06:42:48.418301+0000 (oldest deadline 2024-09-19T06:43:13.718207+0000) 2024-09-19T06:43:17.787 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:17 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:17.535+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:18.233 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:17 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:17.786+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.430806+0000 front 2024-09-19T06:42:48.430819+0000 (oldest deadline 2024-09-19T06:43:10.130557+0000) 2024-09-19T06:43:18.233 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:17 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:17.786+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:54.231865+0000 front 2024-09-19T06:42:54.231770+0000 (oldest deadline 2024-09-19T06:43:17.731624+0000) 2024-09-19T06:43:18.233 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:17 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:17.982+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:44.915444+0000 front 2024-09-19T06:42:44.915317+0000 (oldest deadline 2024-09-19T06:43:10.815165+0000) 2024-09-19T06:43:18.949 INFO:journalctl@ceph.osd.4.smithi155.stdout:Sep 19 06:43:18 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-4[40915]: 2024-09-19T06:43:18.680+0000 7f7d514d2640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.418462+0000 front 2024-09-19T06:42:48.418301+0000 (oldest deadline 2024-09-19T06:43:13.718207+0000) 2024-09-19T06:43:18.950 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:18 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:18.551+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:18.950 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:18 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:18.763+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.430806+0000 front 2024-09-19T06:42:48.430819+0000 (oldest deadline 2024-09-19T06:43:10.130557+0000) 2024-09-19T06:43:18.950 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:18 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:18.763+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:54.231865+0000 front 2024-09-19T06:42:54.231770+0000 (oldest deadline 2024-09-19T06:43:17.731624+0000) 2024-09-19T06:43:19.232 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:18 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:18.948+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:44.915444+0000 front 2024-09-19T06:42:44.915317+0000 (oldest deadline 2024-09-19T06:43:10.815165+0000) 2024-09-19T06:43:19.233 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:18 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:18.948+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:54.916469+0000 front 2024-09-19T06:42:54.916344+0000 (oldest deadline 2024-09-19T06:43:18.416289+0000) 2024-09-19T06:43:19.935 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:19 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:19.553+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:19.936 INFO:journalctl@ceph.osd.4.smithi155.stdout:Sep 19 06:43:19 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-4[40915]: 2024-09-19T06:43:19.633+0000 7f7d514d2640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.418462+0000 front 2024-09-19T06:42:48.418301+0000 (oldest deadline 2024-09-19T06:43:13.718207+0000) 2024-09-19T06:43:19.936 INFO:journalctl@ceph.osd.4.smithi155.stdout:Sep 19 06:43:19 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-4[40915]: 2024-09-19T06:43:19.633+0000 7f7d514d2640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:55.419309+0000 front 2024-09-19T06:42:55.419234+0000 (oldest deadline 2024-09-19T06:43:18.819919+0000) 2024-09-19T06:43:19.936 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:19 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:19.716+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.430806+0000 front 2024-09-19T06:42:48.430819+0000 (oldest deadline 2024-09-19T06:43:10.130557+0000) 2024-09-19T06:43:19.936 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:19 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:19.716+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:54.231865+0000 front 2024-09-19T06:42:54.231770+0000 (oldest deadline 2024-09-19T06:43:17.731624+0000) 2024-09-19T06:43:20.232 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:19 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:19.934+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:44.915444+0000 front 2024-09-19T06:42:44.915317+0000 (oldest deadline 2024-09-19T06:43:10.815165+0000) 2024-09-19T06:43:20.232 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:19 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:19.934+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:54.916469+0000 front 2024-09-19T06:42:54.916344+0000 (oldest deadline 2024-09-19T06:43:18.416289+0000) 2024-09-19T06:43:20.763 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:20 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:20.507+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:20.763 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:20 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:20.507+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:55.708724+0000 front 2024-09-19T06:42:55.708757+0000 (oldest deadline 2024-09-19T06:43:20.408665+0000) 2024-09-19T06:43:20.763 INFO:journalctl@ceph.osd.4.smithi155.stdout:Sep 19 06:43:20 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-4[40915]: 2024-09-19T06:43:20.635+0000 7f7d514d2640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.418462+0000 front 2024-09-19T06:42:48.418301+0000 (oldest deadline 2024-09-19T06:43:13.718207+0000) 2024-09-19T06:43:20.763 INFO:journalctl@ceph.osd.4.smithi155.stdout:Sep 19 06:43:20 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-4[40915]: 2024-09-19T06:43:20.635+0000 7f7d514d2640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:55.419309+0000 front 2024-09-19T06:42:55.419234+0000 (oldest deadline 2024-09-19T06:43:18.819919+0000) 2024-09-19T06:43:21.232 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:20 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:20.761+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.430806+0000 front 2024-09-19T06:42:48.430819+0000 (oldest deadline 2024-09-19T06:43:10.130557+0000) 2024-09-19T06:43:21.233 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:20 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:20.761+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:54.231865+0000 front 2024-09-19T06:42:54.231770+0000 (oldest deadline 2024-09-19T06:43:17.731624+0000) 2024-09-19T06:43:21.233 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:20 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:20.966+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:44.915444+0000 front 2024-09-19T06:42:44.915317+0000 (oldest deadline 2024-09-19T06:43:10.815165+0000) 2024-09-19T06:43:21.233 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:20 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:20.966+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:54.916469+0000 front 2024-09-19T06:42:54.916344+0000 (oldest deadline 2024-09-19T06:43:18.416289+0000) 2024-09-19T06:43:21.536 INFO:journalctl@ceph.osd.4.smithi155.stdout:Sep 19 06:43:21 smithi155 podman[72082]: 2024-09-19 06:43:21.233154844 +0000 UTC m=+5.591157884 container died 7e08cad4b619397290ef2d567645e991f23bd8d0a320f66eefedcbcfd631dbdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:43:21.537 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:21 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:21.462+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:21.537 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:21 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:21.462+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:55.708724+0000 front 2024-09-19T06:42:55.708757+0000 (oldest deadline 2024-09-19T06:43:20.408665+0000) 2024-09-19T06:43:21.857 INFO:journalctl@ceph.osd.4.smithi155.stdout:Sep 19 06:43:21 smithi155 podman[72082]: 2024-09-19 06:43:21.646722174 +0000 UTC m=+6.004725220 container cleanup 7e08cad4b619397290ef2d567645e991f23bd8d0a320f66eefedcbcfd631dbdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:43:21.857 INFO:journalctl@ceph.osd.4.smithi155.stdout:Sep 19 06:43:21 smithi155 bash[72082]: ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-4 2024-09-19T06:43:21.857 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:21 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:21.755+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.430806+0000 front 2024-09-19T06:42:48.430819+0000 (oldest deadline 2024-09-19T06:43:10.130557+0000) 2024-09-19T06:43:21.857 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:21 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:21.755+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:54.231865+0000 front 2024-09-19T06:42:54.231770+0000 (oldest deadline 2024-09-19T06:43:17.731624+0000) 2024-09-19T06:43:22.232 INFO:journalctl@ceph.osd.4.smithi155.stdout:Sep 19 06:43:21 smithi155 podman[72098]: 2024-09-19 06:43:21.85671318 +0000 UTC m=+0.625877617 container remove 7e08cad4b619397290ef2d567645e991f23bd8d0a320f66eefedcbcfd631dbdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:43:22.233 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:21 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:21.953+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:44.915444+0000 front 2024-09-19T06:42:44.915317+0000 (oldest deadline 2024-09-19T06:43:10.815165+0000) 2024-09-19T06:43:22.233 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:21 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:21.953+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:54.916469+0000 front 2024-09-19T06:42:54.916344+0000 (oldest deadline 2024-09-19T06:43:18.416289+0000) 2024-09-19T06:43:22.560 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:22 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:22.454+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:22.561 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:22 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:22.454+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:55.708724+0000 front 2024-09-19T06:42:55.708757+0000 (oldest deadline 2024-09-19T06:43:20.408665+0000) 2024-09-19T06:43:22.561 INFO:journalctl@ceph.osd.4.smithi155.stdout:Sep 19 06:43:22 smithi155 podman[72186]: 2024-09-19 06:43:22.190270866 +0000 UTC m=+0.030834103 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:43:22.561 INFO:journalctl@ceph.osd.4.smithi155.stdout:Sep 19 06:43:22 smithi155 podman[72186]: 2024-09-19 06:43:22.299338565 +0000 UTC m=+0.139901789 container create 57d97a4b20dcee2eacc358356e861903aaba476c9b0d3ce93ff037e823a065a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:43:22.561 INFO:journalctl@ceph.osd.4.smithi155.stdout:Sep 19 06:43:22 smithi155 podman[72186]: 2024-09-19 06:43:22.560269487 +0000 UTC m=+0.400832729 container init 57d97a4b20dcee2eacc358356e861903aaba476c9b0d3ce93ff037e823a065a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4) 2024-09-19T06:43:22.941 INFO:journalctl@ceph.osd.4.smithi155.stdout:Sep 19 06:43:22 smithi155 podman[72186]: 2024-09-19 06:43:22.563801047 +0000 UTC m=+0.404364281 container start 57d97a4b20dcee2eacc358356e861903aaba476c9b0d3ce93ff037e823a065a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-19T06:43:22.941 INFO:journalctl@ceph.osd.4.smithi155.stdout:Sep 19 06:43:22 smithi155 podman[72186]: 2024-09-19 06:43:22.668841193 +0000 UTC m=+0.509404431 container attach 57d97a4b20dcee2eacc358356e861903aaba476c9b0d3ce93ff037e823a065a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-19T06:43:22.941 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:22 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:22.803+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.430806+0000 front 2024-09-19T06:42:48.430819+0000 (oldest deadline 2024-09-19T06:43:10.130557+0000) 2024-09-19T06:43:22.942 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:22 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:22.803+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:54.231865+0000 front 2024-09-19T06:42:54.231770+0000 (oldest deadline 2024-09-19T06:43:17.731624+0000) 2024-09-19T06:43:23.216 INFO:journalctl@ceph.osd.4.smithi155.stdout:Sep 19 06:43:22 smithi155 podman[72186]: 2024-09-19 06:43:22.965041277 +0000 UTC m=+0.805604520 container died 57d97a4b20dcee2eacc358356e861903aaba476c9b0d3ce93ff037e823a065a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-19T06:43:23.216 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:22 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:22.938+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:44.915444+0000 front 2024-09-19T06:42:44.915317+0000 (oldest deadline 2024-09-19T06:43:10.815165+0000) 2024-09-19T06:43:23.217 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:22 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:22.938+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:54.916469+0000 front 2024-09-19T06:42:54.916344+0000 (oldest deadline 2024-09-19T06:43:18.416289+0000) 2024-09-19T06:43:23.570 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f2876d0-764d-11ef-bceb-c7b262605968@osd.4.service' 2024-09-19T06:43:23.651 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:43:23.651 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-19T06:43:23.651 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-19T06:43:23.652 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-9f2876d0-764d-11ef-bceb-c7b262605968@osd.5 2024-09-19T06:43:23.732 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:23 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:23.478+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:23.732 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:23 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:23.478+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:55.708724+0000 front 2024-09-19T06:42:55.708757+0000 (oldest deadline 2024-09-19T06:43:20.408665+0000) 2024-09-19T06:43:24.004 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:23 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:23.983+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:44.915444+0000 front 2024-09-19T06:42:44.915317+0000 (oldest deadline 2024-09-19T06:43:10.815165+0000) 2024-09-19T06:43:24.004 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:23 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:23.983+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:54.916469+0000 front 2024-09-19T06:42:54.916344+0000 (oldest deadline 2024-09-19T06:43:18.416289+0000) 2024-09-19T06:43:24.004 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:23 smithi155 systemd[1]: Stopping Ceph osd.5 for 9f2876d0-764d-11ef-bceb-c7b262605968... 2024-09-19T06:43:24.004 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:23 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:23.785+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.430806+0000 front 2024-09-19T06:42:48.430819+0000 (oldest deadline 2024-09-19T06:43:10.130557+0000) 2024-09-19T06:43:24.005 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:23 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:23.785+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:54.231865+0000 front 2024-09-19T06:42:54.231770+0000 (oldest deadline 2024-09-19T06:43:17.731624+0000) 2024-09-19T06:43:24.005 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:23 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:23.984+0000 7f7e325e3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:43:24.005 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:23 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:23.984+0000 7f7e325e3640 -1 osd.5 72 *** Got signal Terminated *** 2024-09-19T06:43:24.005 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:23 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:23.984+0000 7f7e325e3640 -1 osd.5 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:43:24.801 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:24 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:24.494+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:24.801 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:24 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:24.494+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:55.708724+0000 front 2024-09-19T06:42:55.708757+0000 (oldest deadline 2024-09-19T06:43:20.408665+0000) 2024-09-19T06:43:25.232 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:24 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:24.800+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.430806+0000 front 2024-09-19T06:42:48.430819+0000 (oldest deadline 2024-09-19T06:43:10.130557+0000) 2024-09-19T06:43:25.233 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:24 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:24.800+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:54.231865+0000 front 2024-09-19T06:42:54.231770+0000 (oldest deadline 2024-09-19T06:43:17.731624+0000) 2024-09-19T06:43:25.233 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:24 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:24.987+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:44.915444+0000 front 2024-09-19T06:42:44.915317+0000 (oldest deadline 2024-09-19T06:43:10.815165+0000) 2024-09-19T06:43:25.233 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:24 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:24.987+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:54.916469+0000 front 2024-09-19T06:42:54.916344+0000 (oldest deadline 2024-09-19T06:43:18.416289+0000) 2024-09-19T06:43:25.732 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:25 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:25.478+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:25.732 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:25 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:25.478+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:55.708724+0000 front 2024-09-19T06:42:55.708757+0000 (oldest deadline 2024-09-19T06:43:20.408665+0000) 2024-09-19T06:43:26.233 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:25 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:25.782+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.430806+0000 front 2024-09-19T06:42:48.430819+0000 (oldest deadline 2024-09-19T06:43:10.130557+0000) 2024-09-19T06:43:26.233 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:25 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:25.782+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:54.231865+0000 front 2024-09-19T06:42:54.231770+0000 (oldest deadline 2024-09-19T06:43:17.731624+0000) 2024-09-19T06:43:26.233 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:25 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:25.979+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:44.915444+0000 front 2024-09-19T06:42:44.915317+0000 (oldest deadline 2024-09-19T06:43:10.815165+0000) 2024-09-19T06:43:26.233 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:25 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:25.979+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:54.916469+0000 front 2024-09-19T06:42:54.916344+0000 (oldest deadline 2024-09-19T06:43:18.416289+0000) 2024-09-19T06:43:26.733 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:26 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:26.470+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:26.733 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:26 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:26.470+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:55.708724+0000 front 2024-09-19T06:42:55.708757+0000 (oldest deadline 2024-09-19T06:43:20.408665+0000) 2024-09-19T06:43:26.733 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:26 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:26.470+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6822 osd.2 since back 2024-09-19T06:43:03.210283+0000 front 2024-09-19T06:43:03.210256+0000 (oldest deadline 2024-09-19T06:43:25.510107+0000) 2024-09-19T06:43:27.232 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:26 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:26.820+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.430806+0000 front 2024-09-19T06:42:48.430819+0000 (oldest deadline 2024-09-19T06:43:10.130557+0000) 2024-09-19T06:43:27.233 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:26 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:26.820+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:54.231865+0000 front 2024-09-19T06:42:54.231770+0000 (oldest deadline 2024-09-19T06:43:17.731624+0000) 2024-09-19T06:43:27.233 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:27 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:27.009+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:44.915444+0000 front 2024-09-19T06:42:44.915317+0000 (oldest deadline 2024-09-19T06:43:10.815165+0000) 2024-09-19T06:43:27.233 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:27 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:27.009+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:54.916469+0000 front 2024-09-19T06:42:54.916344+0000 (oldest deadline 2024-09-19T06:43:18.416289+0000) 2024-09-19T06:43:27.732 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:27 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:27.469+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:27.733 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:27 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:27.469+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:55.708724+0000 front 2024-09-19T06:42:55.708757+0000 (oldest deadline 2024-09-19T06:43:20.408665+0000) 2024-09-19T06:43:27.733 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:27 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:27.469+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6822 osd.2 since back 2024-09-19T06:43:03.210283+0000 front 2024-09-19T06:43:03.210256+0000 (oldest deadline 2024-09-19T06:43:25.510107+0000) 2024-09-19T06:43:28.045 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:27 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:27.779+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.430806+0000 front 2024-09-19T06:42:48.430819+0000 (oldest deadline 2024-09-19T06:43:10.130557+0000) 2024-09-19T06:43:28.045 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:27 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:27.779+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:54.231865+0000 front 2024-09-19T06:42:54.231770+0000 (oldest deadline 2024-09-19T06:43:17.731624+0000) 2024-09-19T06:43:28.045 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:27 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:27.779+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6822 osd.2 since back 2024-09-19T06:43:02.432761+0000 front 2024-09-19T06:43:02.432696+0000 (oldest deadline 2024-09-19T06:43:27.132554+0000) 2024-09-19T06:43:28.424 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:28 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:28.044+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:44.915444+0000 front 2024-09-19T06:42:44.915317+0000 (oldest deadline 2024-09-19T06:43:10.815165+0000) 2024-09-19T06:43:28.424 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:28 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:28.044+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:54.916469+0000 front 2024-09-19T06:42:54.916344+0000 (oldest deadline 2024-09-19T06:43:18.416289+0000) 2024-09-19T06:43:28.424 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:28 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:28.044+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6822 osd.2 since back 2024-09-19T06:43:01.317253+0000 front 2024-09-19T06:43:01.317409+0000 (oldest deadline 2024-09-19T06:43:27.217364+0000) 2024-09-19T06:43:28.732 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:28 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:28.423+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:28.732 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:28 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:28.423+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:55.708724+0000 front 2024-09-19T06:42:55.708757+0000 (oldest deadline 2024-09-19T06:43:20.408665+0000) 2024-09-19T06:43:28.732 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:28 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:28.423+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6822 osd.2 since back 2024-09-19T06:43:03.210283+0000 front 2024-09-19T06:43:03.210256+0000 (oldest deadline 2024-09-19T06:43:25.510107+0000) 2024-09-19T06:43:29.031 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:28 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:28.821+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.430806+0000 front 2024-09-19T06:42:48.430819+0000 (oldest deadline 2024-09-19T06:43:10.130557+0000) 2024-09-19T06:43:29.031 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:28 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:28.821+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:54.231865+0000 front 2024-09-19T06:42:54.231770+0000 (oldest deadline 2024-09-19T06:43:17.731624+0000) 2024-09-19T06:43:29.032 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:28 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5[46832]: 2024-09-19T06:43:28.821+0000 7f7e2abf1640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.53:6822 osd.2 since back 2024-09-19T06:43:02.432761+0000 front 2024-09-19T06:43:02.432696+0000 (oldest deadline 2024-09-19T06:43:27.132554+0000) 2024-09-19T06:43:29.378 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:29 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:29.030+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:44.915444+0000 front 2024-09-19T06:42:44.915317+0000 (oldest deadline 2024-09-19T06:43:10.815165+0000) 2024-09-19T06:43:29.378 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:29 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:29.030+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:54.916469+0000 front 2024-09-19T06:42:54.916344+0000 (oldest deadline 2024-09-19T06:43:18.416289+0000) 2024-09-19T06:43:29.379 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:29 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:29.030+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6822 osd.2 since back 2024-09-19T06:43:01.317253+0000 front 2024-09-19T06:43:01.317409+0000 (oldest deadline 2024-09-19T06:43:27.217364+0000) 2024-09-19T06:43:29.732 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:29 smithi155 podman[72292]: 2024-09-19 06:43:29.544012989 +0000 UTC m=+5.660748487 container died 1c97b1724d7ee38b436602bc1c13be3da40fd3b9754388c1a51134461b8dfe33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:43:29.733 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:29 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:29.378+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:29.733 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:29 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:29.378+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:55.708724+0000 front 2024-09-19T06:42:55.708757+0000 (oldest deadline 2024-09-19T06:43:20.408665+0000) 2024-09-19T06:43:29.733 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:29 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:29.378+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6822 osd.2 since back 2024-09-19T06:43:03.210283+0000 front 2024-09-19T06:43:03.210256+0000 (oldest deadline 2024-09-19T06:43:25.510107+0000) 2024-09-19T06:43:30.136 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:29 smithi155 podman[72292]: 2024-09-19 06:43:29.943017439 +0000 UTC m=+6.059752936 container cleanup 1c97b1724d7ee38b436602bc1c13be3da40fd3b9754388c1a51134461b8dfe33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-19T06:43:30.137 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:29 smithi155 bash[72292]: ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5 2024-09-19T06:43:30.137 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:29 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:29.991+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:44.915444+0000 front 2024-09-19T06:42:44.915317+0000 (oldest deadline 2024-09-19T06:43:10.815165+0000) 2024-09-19T06:43:30.137 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:29 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:29.991+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:54.916469+0000 front 2024-09-19T06:42:54.916344+0000 (oldest deadline 2024-09-19T06:43:18.416289+0000) 2024-09-19T06:43:30.137 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:29 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:29.991+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6822 osd.2 since back 2024-09-19T06:43:01.317253+0000 front 2024-09-19T06:43:01.317409+0000 (oldest deadline 2024-09-19T06:43:27.217364+0000) 2024-09-19T06:43:30.395 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:30 smithi155 podman[72310]: 2024-09-19 06:43:30.136228148 +0000 UTC m=+0.586602693 container remove 1c97b1724d7ee38b436602bc1c13be3da40fd3b9754388c1a51134461b8dfe33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:43:30.680 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:30 smithi155 podman[72396]: 2024-09-19 06:43:30.461719504 +0000 UTC m=+0.016143184 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:43:30.680 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:30 smithi155 podman[72396]: 2024-09-19 06:43:30.562154178 +0000 UTC m=+0.116577858 container create 9e260171162307eef5089733359dc80dd6b39f14b8b844571ee16bbffc2bac9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-19T06:43:30.680 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:30 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:30.394+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:30.680 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:30 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:30.394+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:55.708724+0000 front 2024-09-19T06:42:55.708757+0000 (oldest deadline 2024-09-19T06:43:20.408665+0000) 2024-09-19T06:43:30.681 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:30 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:30.394+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6822 osd.2 since back 2024-09-19T06:43:03.210283+0000 front 2024-09-19T06:43:03.210256+0000 (oldest deadline 2024-09-19T06:43:25.510107+0000) 2024-09-19T06:43:30.972 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:30 smithi155 podman[72396]: 2024-09-19 06:43:30.806748975 +0000 UTC m=+0.361172656 container init 9e260171162307eef5089733359dc80dd6b39f14b8b844571ee16bbffc2bac9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-19T06:43:30.972 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:30 smithi155 podman[72396]: 2024-09-19 06:43:30.810047852 +0000 UTC m=+0.364471542 container start 9e260171162307eef5089733359dc80dd6b39f14b8b844571ee16bbffc2bac9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-19T06:43:30.972 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:30 smithi155 podman[72396]: 2024-09-19 06:43:30.87339353 +0000 UTC m=+0.427817223 container attach 9e260171162307eef5089733359dc80dd6b39f14b8b844571ee16bbffc2bac9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:43:31.232 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 19 06:43:31 smithi155 podman[72396]: 2024-09-19 06:43:31.000580163 +0000 UTC m=+0.555003857 container died 9e260171162307eef5089733359dc80dd6b39f14b8b844571ee16bbffc2bac9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-5-deactivate, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-19T06:43:31.233 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:30 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:30.970+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:44.915444+0000 front 2024-09-19T06:42:44.915317+0000 (oldest deadline 2024-09-19T06:43:10.815165+0000) 2024-09-19T06:43:31.233 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:30 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:30.970+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:54.916469+0000 front 2024-09-19T06:42:54.916344+0000 (oldest deadline 2024-09-19T06:43:18.416289+0000) 2024-09-19T06:43:31.233 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:30 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:30.970+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6822 osd.2 since back 2024-09-19T06:43:01.317253+0000 front 2024-09-19T06:43:01.317409+0000 (oldest deadline 2024-09-19T06:43:27.217364+0000) 2024-09-19T06:43:31.586 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:31 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:31.370+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:31.586 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:31 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:31.370+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:55.708724+0000 front 2024-09-19T06:42:55.708757+0000 (oldest deadline 2024-09-19T06:43:20.408665+0000) 2024-09-19T06:43:31.586 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:31 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:31.370+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6822 osd.2 since back 2024-09-19T06:43:03.210283+0000 front 2024-09-19T06:43:03.210256+0000 (oldest deadline 2024-09-19T06:43:25.510107+0000) 2024-09-19T06:43:31.605 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f2876d0-764d-11ef-bceb-c7b262605968@osd.5.service' 2024-09-19T06:43:31.687 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:43:31.688 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-19T06:43:31.688 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-19T06:43:31.688 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-9f2876d0-764d-11ef-bceb-c7b262605968@osd.6 2024-09-19T06:43:31.982 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:31 smithi155 systemd[1]: Stopping Ceph osd.6 for 9f2876d0-764d-11ef-bceb-c7b262605968... 2024-09-19T06:43:32.353 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:32 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:32.010+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:44.915444+0000 front 2024-09-19T06:42:44.915317+0000 (oldest deadline 2024-09-19T06:43:10.815165+0000) 2024-09-19T06:43:32.353 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:32 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:32.010+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:54.916469+0000 front 2024-09-19T06:42:54.916344+0000 (oldest deadline 2024-09-19T06:43:18.416289+0000) 2024-09-19T06:43:32.353 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:32 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:32.010+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6822 osd.2 since back 2024-09-19T06:43:01.317253+0000 front 2024-09-19T06:43:01.317409+0000 (oldest deadline 2024-09-19T06:43:27.217364+0000) 2024-09-19T06:43:32.353 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:32 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:32.047+0000 7fef4ea29640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:43:32.353 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:32 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:32.047+0000 7fef4ea29640 -1 osd.6 72 *** Got signal Terminated *** 2024-09-19T06:43:32.353 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:32 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:32.047+0000 7fef4ea29640 -1 osd.6 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:43:32.732 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:32 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:32.352+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:32.732 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:32 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:32.352+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:55.708724+0000 front 2024-09-19T06:42:55.708757+0000 (oldest deadline 2024-09-19T06:43:20.408665+0000) 2024-09-19T06:43:32.732 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:32 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:32.352+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6822 osd.2 since back 2024-09-19T06:43:03.210283+0000 front 2024-09-19T06:43:03.210256+0000 (oldest deadline 2024-09-19T06:43:25.510107+0000) 2024-09-19T06:43:33.306 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:33 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:33.048+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:44.915444+0000 front 2024-09-19T06:42:44.915317+0000 (oldest deadline 2024-09-19T06:43:10.815165+0000) 2024-09-19T06:43:33.306 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:33 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:33.048+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:54.916469+0000 front 2024-09-19T06:42:54.916344+0000 (oldest deadline 2024-09-19T06:43:18.416289+0000) 2024-09-19T06:43:33.306 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:33 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:33.048+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6822 osd.2 since back 2024-09-19T06:43:01.317253+0000 front 2024-09-19T06:43:01.317409+0000 (oldest deadline 2024-09-19T06:43:27.217364+0000) 2024-09-19T06:43:33.732 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:33 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:33.304+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:33.733 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:33 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:33.304+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:55.708724+0000 front 2024-09-19T06:42:55.708757+0000 (oldest deadline 2024-09-19T06:43:20.408665+0000) 2024-09-19T06:43:33.733 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:33 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:33.304+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6822 osd.2 since back 2024-09-19T06:43:03.210283+0000 front 2024-09-19T06:43:03.210256+0000 (oldest deadline 2024-09-19T06:43:25.510107+0000) 2024-09-19T06:43:34.482 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:34 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:34.060+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:44.915444+0000 front 2024-09-19T06:42:44.915317+0000 (oldest deadline 2024-09-19T06:43:10.815165+0000) 2024-09-19T06:43:34.482 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:34 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:34.060+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:54.916469+0000 front 2024-09-19T06:42:54.916344+0000 (oldest deadline 2024-09-19T06:43:18.416289+0000) 2024-09-19T06:43:34.483 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:34 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:34.060+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6822 osd.2 since back 2024-09-19T06:43:01.317253+0000 front 2024-09-19T06:43:01.317409+0000 (oldest deadline 2024-09-19T06:43:27.217364+0000) 2024-09-19T06:43:34.483 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:34 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:34.256+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:34.483 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:34 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:34.256+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:55.708724+0000 front 2024-09-19T06:42:55.708757+0000 (oldest deadline 2024-09-19T06:43:20.408665+0000) 2024-09-19T06:43:34.483 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:34 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:34.256+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6822 osd.2 since back 2024-09-19T06:43:03.210283+0000 front 2024-09-19T06:43:03.210256+0000 (oldest deadline 2024-09-19T06:43:25.510107+0000) 2024-09-19T06:43:35.482 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:35 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:35.055+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:44.915444+0000 front 2024-09-19T06:42:44.915317+0000 (oldest deadline 2024-09-19T06:43:10.815165+0000) 2024-09-19T06:43:35.482 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:35 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:35.055+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:54.916469+0000 front 2024-09-19T06:42:54.916344+0000 (oldest deadline 2024-09-19T06:43:18.416289+0000) 2024-09-19T06:43:35.482 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:35 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:35.055+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6822 osd.2 since back 2024-09-19T06:43:01.317253+0000 front 2024-09-19T06:43:01.317409+0000 (oldest deadline 2024-09-19T06:43:27.217364+0000) 2024-09-19T06:43:35.483 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:35 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:35.239+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:35.483 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:35 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:35.239+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:55.708724+0000 front 2024-09-19T06:42:55.708757+0000 (oldest deadline 2024-09-19T06:43:20.408665+0000) 2024-09-19T06:43:35.483 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:35 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:35.239+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6822 osd.2 since back 2024-09-19T06:43:03.210283+0000 front 2024-09-19T06:43:03.210256+0000 (oldest deadline 2024-09-19T06:43:25.510107+0000) 2024-09-19T06:43:36.482 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:36 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:36.013+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:44.915444+0000 front 2024-09-19T06:42:44.915317+0000 (oldest deadline 2024-09-19T06:43:10.815165+0000) 2024-09-19T06:43:36.482 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:36 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:36.013+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:54.916469+0000 front 2024-09-19T06:42:54.916344+0000 (oldest deadline 2024-09-19T06:43:18.416289+0000) 2024-09-19T06:43:36.482 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:36 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:36.013+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6822 osd.2 since back 2024-09-19T06:43:01.317253+0000 front 2024-09-19T06:43:01.317409+0000 (oldest deadline 2024-09-19T06:43:27.217364+0000) 2024-09-19T06:43:36.483 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:36 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:36.192+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:36.483 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:36 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:36.192+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:55.708724+0000 front 2024-09-19T06:42:55.708757+0000 (oldest deadline 2024-09-19T06:43:20.408665+0000) 2024-09-19T06:43:36.483 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:36 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:36.192+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6822 osd.2 since back 2024-09-19T06:43:03.210283+0000 front 2024-09-19T06:43:03.210256+0000 (oldest deadline 2024-09-19T06:43:25.510107+0000) 2024-09-19T06:43:37.482 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:37 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:37.037+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:44.915444+0000 front 2024-09-19T06:42:44.915317+0000 (oldest deadline 2024-09-19T06:43:10.815165+0000) 2024-09-19T06:43:37.483 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:37 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:37.037+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:54.916469+0000 front 2024-09-19T06:42:54.916344+0000 (oldest deadline 2024-09-19T06:43:18.416289+0000) 2024-09-19T06:43:37.483 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:37 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6[53092]: 2024-09-19T06:43:37.037+0000 7fef4b03f640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.53:6822 osd.2 since back 2024-09-19T06:43:01.317253+0000 front 2024-09-19T06:43:01.317409+0000 (oldest deadline 2024-09-19T06:43:27.217364+0000) 2024-09-19T06:43:37.483 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:37 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:37.187+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:37.483 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:37 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:37.187+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:55.708724+0000 front 2024-09-19T06:42:55.708757+0000 (oldest deadline 2024-09-19T06:43:20.408665+0000) 2024-09-19T06:43:37.483 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:37 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:37.187+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6822 osd.2 since back 2024-09-19T06:43:03.210283+0000 front 2024-09-19T06:43:03.210256+0000 (oldest deadline 2024-09-19T06:43:25.510107+0000) 2024-09-19T06:43:37.845 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:37 smithi155 podman[72498]: 2024-09-19 06:43:37.572382136 +0000 UTC m=+5.649465627 container died 8a895f8b6ce682b4f76efb8c5d3a17e3a3738d0a96324941953131df06c58b94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-19T06:43:38.229 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:37 smithi155 podman[72498]: 2024-09-19 06:43:37.943779215 +0000 UTC m=+6.020862699 container cleanup 8a895f8b6ce682b4f76efb8c5d3a17e3a3738d0a96324941953131df06c58b94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-19T06:43:38.229 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:37 smithi155 bash[72498]: ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6 2024-09-19T06:43:38.482 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:38 smithi155 podman[72514]: 2024-09-19 06:43:38.228545245 +0000 UTC m=+0.656641370 container remove 8a895f8b6ce682b4f76efb8c5d3a17e3a3738d0a96324941953131df06c58b94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, ceph=True) 2024-09-19T06:43:38.482 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:38 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:38.236+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:38.482 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:38 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:38.236+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:55.708724+0000 front 2024-09-19T06:42:55.708757+0000 (oldest deadline 2024-09-19T06:43:20.408665+0000) 2024-09-19T06:43:38.483 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:38 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:38.236+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6822 osd.2 since back 2024-09-19T06:43:03.210283+0000 front 2024-09-19T06:43:03.210256+0000 (oldest deadline 2024-09-19T06:43:25.510107+0000) 2024-09-19T06:43:38.849 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:38 smithi155 podman[72605]: 2024-09-19 06:43:38.467889445 +0000 UTC m=+0.016070831 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:43:38.850 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:38 smithi155 podman[72605]: 2024-09-19 06:43:38.604662036 +0000 UTC m=+0.152843423 container create 96a410c8fdf5871b34e4134a0381acf7e6675d9c5555cc4ce72cf4ac3bdd6cee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-09-19T06:43:38.850 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:38 smithi155 podman[72605]: 2024-09-19 06:43:38.849140977 +0000 UTC m=+0.397322373 container init 96a410c8fdf5871b34e4134a0381acf7e6675d9c5555cc4ce72cf4ac3bdd6cee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True) 2024-09-19T06:43:39.232 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:38 smithi155 podman[72605]: 2024-09-19 06:43:38.852484002 +0000 UTC m=+0.400665390 container start 96a410c8fdf5871b34e4134a0381acf7e6675d9c5555cc4ce72cf4ac3bdd6cee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-19T06:43:39.233 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:38 smithi155 podman[72605]: 2024-09-19 06:43:38.965910775 +0000 UTC m=+0.514092171 container attach 96a410c8fdf5871b34e4134a0381acf7e6675d9c5555cc4ce72cf4ac3bdd6cee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:43:39.233 INFO:journalctl@ceph.osd.6.smithi155.stdout:Sep 19 06:43:39 smithi155 podman[72605]: 2024-09-19 06:43:39.039681573 +0000 UTC m=+0.587862968 container died 96a410c8fdf5871b34e4134a0381acf7e6675d9c5555cc4ce72cf4ac3bdd6cee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-19T06:43:39.617 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:39 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:39.250+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:39.617 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:39 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:39.250+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:55.708724+0000 front 2024-09-19T06:42:55.708757+0000 (oldest deadline 2024-09-19T06:43:20.408665+0000) 2024-09-19T06:43:39.617 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:39 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:39.250+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6822 osd.2 since back 2024-09-19T06:43:03.210283+0000 front 2024-09-19T06:43:03.210256+0000 (oldest deadline 2024-09-19T06:43:25.510107+0000) 2024-09-19T06:43:39.617 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:39 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:39.250+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6830 osd.3 since back 2024-09-19T06:43:10.811239+0000 front 2024-09-19T06:43:10.811247+0000 (oldest deadline 2024-09-19T06:43:39.011443+0000) 2024-09-19T06:43:39.640 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f2876d0-764d-11ef-bceb-c7b262605968@osd.6.service' 2024-09-19T06:43:39.678 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:43:39.678 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-19T06:43:39.678 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-19T06:43:39.678 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-9f2876d0-764d-11ef-bceb-c7b262605968@osd.7 2024-09-19T06:43:39.982 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:39 smithi155 systemd[1]: Stopping Ceph osd.7 for 9f2876d0-764d-11ef-bceb-c7b262605968... 2024-09-19T06:43:40.482 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:39 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:39.984+0000 7fe0ff795640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:43:40.483 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:39 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:39.984+0000 7fe0ff795640 -1 osd.7 72 *** Got signal Terminated *** 2024-09-19T06:43:40.483 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:39 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:39.984+0000 7fe0ff795640 -1 osd.7 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:43:40.483 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:40 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:40.201+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:40.483 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:40 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:40.201+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:55.708724+0000 front 2024-09-19T06:42:55.708757+0000 (oldest deadline 2024-09-19T06:43:20.408665+0000) 2024-09-19T06:43:40.483 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:40 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:40.201+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6822 osd.2 since back 2024-09-19T06:43:03.210283+0000 front 2024-09-19T06:43:03.210256+0000 (oldest deadline 2024-09-19T06:43:25.510107+0000) 2024-09-19T06:43:40.483 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:40 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:40.201+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6830 osd.3 since back 2024-09-19T06:43:10.811239+0000 front 2024-09-19T06:43:10.811247+0000 (oldest deadline 2024-09-19T06:43:39.011443+0000) 2024-09-19T06:43:41.482 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:41 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:41.202+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:41.483 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:41 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:41.202+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:55.708724+0000 front 2024-09-19T06:42:55.708757+0000 (oldest deadline 2024-09-19T06:43:20.408665+0000) 2024-09-19T06:43:41.483 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:41 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:41.202+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6822 osd.2 since back 2024-09-19T06:43:03.210283+0000 front 2024-09-19T06:43:03.210256+0000 (oldest deadline 2024-09-19T06:43:25.510107+0000) 2024-09-19T06:43:41.483 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:41 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:41.202+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6830 osd.3 since back 2024-09-19T06:43:10.811239+0000 front 2024-09-19T06:43:10.811247+0000 (oldest deadline 2024-09-19T06:43:39.011443+0000) 2024-09-19T06:43:42.482 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:42 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:42.224+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:42.482 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:42 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:42.224+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:55.708724+0000 front 2024-09-19T06:42:55.708757+0000 (oldest deadline 2024-09-19T06:43:20.408665+0000) 2024-09-19T06:43:42.482 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:42 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:42.224+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6822 osd.2 since back 2024-09-19T06:43:03.210283+0000 front 2024-09-19T06:43:03.210256+0000 (oldest deadline 2024-09-19T06:43:25.510107+0000) 2024-09-19T06:43:42.483 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:42 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:42.224+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6830 osd.3 since back 2024-09-19T06:43:10.811239+0000 front 2024-09-19T06:43:10.811247+0000 (oldest deadline 2024-09-19T06:43:39.011443+0000) 2024-09-19T06:43:43.627 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:43 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:43.242+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:43.628 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:43 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:43.242+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:55.708724+0000 front 2024-09-19T06:42:55.708757+0000 (oldest deadline 2024-09-19T06:43:20.408665+0000) 2024-09-19T06:43:43.628 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:43 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:43.242+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6822 osd.2 since back 2024-09-19T06:43:03.210283+0000 front 2024-09-19T06:43:03.210256+0000 (oldest deadline 2024-09-19T06:43:25.510107+0000) 2024-09-19T06:43:43.628 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:43 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:43.242+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6830 osd.3 since back 2024-09-19T06:43:10.811239+0000 front 2024-09-19T06:43:10.811247+0000 (oldest deadline 2024-09-19T06:43:39.011443+0000) 2024-09-19T06:43:44.482 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:44 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:44.215+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-09-19T06:42:48.206761+0000 front 2024-09-19T06:42:48.206684+0000 (oldest deadline 2024-09-19T06:43:12.306854+0000) 2024-09-19T06:43:44.482 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:44 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:44.215+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6814 osd.1 since back 2024-09-19T06:42:55.708724+0000 front 2024-09-19T06:42:55.708757+0000 (oldest deadline 2024-09-19T06:43:20.408665+0000) 2024-09-19T06:43:44.483 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:44 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:44.215+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6822 osd.2 since back 2024-09-19T06:43:03.210283+0000 front 2024-09-19T06:43:03.210256+0000 (oldest deadline 2024-09-19T06:43:25.510107+0000) 2024-09-19T06:43:44.483 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:44 smithi155 ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7[58963]: 2024-09-19T06:43:44.215+0000 7fe0f7da3640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.53:6830 osd.3 since back 2024-09-19T06:43:10.811239+0000 front 2024-09-19T06:43:10.811247+0000 (oldest deadline 2024-09-19T06:43:39.011443+0000) 2024-09-19T06:43:45.798 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:45 smithi155 podman[72707]: 2024-09-19 06:43:45.509805391 +0000 UTC m=+5.610319679 container died 38847310f49915feb95f3c10fd7dea716961a90c4e29da6af0d3fa171e66ee42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-19T06:43:46.214 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:45 smithi155 podman[72707]: 2024-09-19 06:43:45.895611531 +0000 UTC m=+5.996125829 container cleanup 38847310f49915feb95f3c10fd7dea716961a90c4e29da6af0d3fa171e66ee42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4) 2024-09-19T06:43:46.214 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:45 smithi155 bash[72707]: ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7 2024-09-19T06:43:46.482 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:46 smithi155 podman[72734]: 2024-09-19 06:43:46.214292926 +0000 UTC m=+0.693017329 container remove 38847310f49915feb95f3c10fd7dea716961a90c4e29da6af0d3fa171e66ee42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-19T06:43:46.821 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:46 smithi155 podman[72821]: 2024-09-19 06:43:46.471293074 +0000 UTC m=+0.015693828 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:43:46.822 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:46 smithi155 podman[72821]: 2024-09-19 06:43:46.615641144 +0000 UTC m=+0.160041875 container create 0c3514fef8e109b2196f2056a3e23b5571850fc3cd4ca4d0ea0b0c211b7450ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-19T06:43:46.822 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:46 smithi155 podman[72821]: 2024-09-19 06:43:46.817986881 +0000 UTC m=+0.362387628 container init 0c3514fef8e109b2196f2056a3e23b5571850fc3cd4ca4d0ea0b0c211b7450ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, io.buildah.version=1.37.2) 2024-09-19T06:43:46.822 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:46 smithi155 podman[72821]: 2024-09-19 06:43:46.821298582 +0000 UTC m=+0.365699309 container start 0c3514fef8e109b2196f2056a3e23b5571850fc3cd4ca4d0ea0b0c211b7450ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-19T06:43:47.233 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:46 smithi155 podman[72821]: 2024-09-19 06:43:46.908959046 +0000 UTC m=+0.453359786 container attach 0c3514fef8e109b2196f2056a3e23b5571850fc3cd4ca4d0ea0b0c211b7450ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7-deactivate, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-19T06:43:47.233 INFO:journalctl@ceph.osd.7.smithi155.stdout:Sep 19 06:43:47 smithi155 podman[72821]: 2024-09-19 06:43:47.019414852 +0000 UTC m=+0.563815592 container died 0c3514fef8e109b2196f2056a3e23b5571850fc3cd4ca4d0ea0b0c211b7450ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9f2876d0-764d-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:43:47.624 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f2876d0-764d-11ef-bceb-c7b262605968@osd.7.service' 2024-09-19T06:43:47.663 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:43:47.663 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-19T06:43:47.664 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T06:43:47.878 INFO:teuthology.orchestra.run.smithi053.stdout:Deleting cluster with fsid: 9f2876d0-764d-11ef-bceb-c7b262605968 2024-09-19T06:44:11.244 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T06:44:11.446 INFO:teuthology.orchestra.run.smithi155.stdout:Deleting cluster with fsid: 9f2876d0-764d-11ef-bceb-c7b262605968 2024-09-19T06:44:25.544 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T06:44:25.579 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T06:44:25.606 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-19T06:44:25.609 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911719/remote/smithi053/crash 2024-09-19T06:44:25.609 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/crash -- . 2024-09-19T06:44:25.664 INFO:teuthology.orchestra.run.smithi053.stderr:tar: /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T06:44:25.665 INFO:teuthology.orchestra.run.smithi053.stderr:tar: Error is not recoverable: exiting now 2024-09-19T06:44:25.667 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911719/remote/smithi155/crash 2024-09-19T06:44:25.667 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/crash -- . 2024-09-19T06:44:25.709 INFO:teuthology.orchestra.run.smithi155.stderr:tar: /var/lib/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T06:44:25.709 INFO:teuthology.orchestra.run.smithi155.stderr:tar: Error is not recoverable: exiting now 2024-09-19T06:44:25.711 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-19T06:44:25.711 DEBUG:teuthology.orchestra.run.smithi053:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-19T06:44:25.809 INFO:tasks.cephadm:Compressing logs... 2024-09-19T06:44:25.809 DEBUG:teuthology.orchestra.run.smithi053:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T06:44:25.811 DEBUG:teuthology.orchestra.run.smithi155:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T06:44:25.837 INFO:teuthology.orchestra.run.smithi155.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-19T06:44:25.837 INFO:teuthology.orchestra.run.smithi155.stderr:: No such file or directory 2024-09-19T06:44:25.837 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T06:44:25.837 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-19T06:44:25.837 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T06:44:25.837 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-mgr.smithi155.piymgv.log 2024-09-19T06:44:25.838 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T06:44:25.838 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-mgr.smithi155.piymgv.log: gzip -5 --verbose -- /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-mds.bar.smithi155.foldmc.log 2024-09-19T06:44:25.840 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-19T06:44:25.840 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-mds.bar.smithi155.foldmc.log: gzip -5 --verbose -- /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-19T06:44:25.840 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph.log 2024-09-19T06:44:25.840 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-client.ceph-exporter.smithi155.log 2024-09-19T06:44:25.840 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-mds.foo.smithi155.dsbavw.log 2024-09-19T06:44:25.840 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-client.ceph-exporter.smithi155.log: gzip -5 --verbose -- /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-mon.smithi155.log 2024-09-19T06:44:25.841 INFO:teuthology.orchestra.run.smithi155.stderr: 68.3% -- replaced with /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-client.ceph-exporter.smithi155.log.gz 2024-09-19T06:44:25.841 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-mds.foo.smithi155.dsbavw.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-09-19T06:44:25.841 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-mon.smithi155.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T06:44:25.847 INFO:teuthology.orchestra.run.smithi053.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-19T06:44:25.847 INFO:teuthology.orchestra.run.smithi053.stderr:: No such file or directory 2024-09-19T06:44:25.847 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-19T06:44:25.848 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T06:44:25.848 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-mds.bar.smithi053.nzdkgr.log 2024-09-19T06:44:25.849 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-client.ceph-exporter.smithi053.log 2024-09-19T06:44:25.849 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-mds.bar.smithi053.nzdkgr.log: gzip -5 --verbose -- /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T06:44:25.849 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-client.ceph-exporter.smithi053.log: gzip -5 --verbose -- /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-mgr.smithi053.trakvg.log 2024-09-19T06:44:25.851 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-mon.smithi053.log 2024-09-19T06:44:25.851 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-mgr.smithi053.trakvg.log: gzip -5 --verbose -- /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T06:44:25.851 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-mon.smithi053.log: gzip -5 --verbose -- /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-19T06:44:25.851 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph.log 2024-09-19T06:44:25.851 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-19T06:44:25.851 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-mds.foo.smithi053.rdifzk.log 2024-09-19T06:44:25.851 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-09-19T06:44:25.852 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-mds.foo.smithi053.rdifzk.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T06:44:25.852 INFO:teuthology.orchestra.run.smithi053.stderr: 94.4% -- replaced with /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-client.ceph-exporter.smithi053.log.gz 2024-09-19T06:44:25.853 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.log: /var/log/ceph/cephadm.log: 88.3% -- replaced with /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T06:44:25.885 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.log: /var/log/ceph/cephadm.log: 83.9% -- replaced with /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T06:44:26.027 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-09-19T06:44:26.056 INFO:teuthology.orchestra.run.smithi053.stderr: 91.6% -- replaced with /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T06:44:26.069 INFO:teuthology.orchestra.run.smithi053.stderr: 84.9% -- replaced with /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T06:44:26.076 INFO:teuthology.orchestra.run.smithi155.stderr: 92.5% -- replaced with /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-mds.foo.smithi155.dsbavw.log.gz 2024-09-19T06:44:26.092 INFO:teuthology.orchestra.run.smithi155.stderr: 90.8% -- replaced with /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T06:44:26.136 INFO:teuthology.orchestra.run.smithi155.stderr: 93.7% -- replaced with /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-mgr.smithi155.piymgv.log.gz 2024-09-19T06:44:26.228 INFO:teuthology.orchestra.run.smithi053.stderr: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T06:44:26.295 INFO:teuthology.orchestra.run.smithi053.stderr: 91.0% -- replaced with /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T06:44:26.361 INFO:teuthology.orchestra.run.smithi155.stderr: 92.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T06:44:26.376 INFO:teuthology.orchestra.run.smithi155.stderr: 91.9% -- replaced with /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T06:44:26.515 INFO:teuthology.orchestra.run.smithi155.stderr: 94.8% -- replaced with /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-mds.bar.smithi155.foldmc.log.gz 2024-09-19T06:44:27.024 INFO:teuthology.orchestra.run.smithi053.stderr: 94.7% -- replaced with /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-mds.foo.smithi053.rdifzk.log.gz 2024-09-19T06:44:27.410 INFO:teuthology.orchestra.run.smithi155.stderr: 88.4% -- replaced with /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T06:44:30.864 INFO:teuthology.orchestra.run.smithi053.stderr: 90.3% -- replaced with /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-mgr.smithi053.trakvg.log.gz 2024-09-19T06:44:45.353 INFO:teuthology.orchestra.run.smithi053.stderr: 90.2% -- replaced with /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-mon.smithi053.log.gz 2024-09-19T06:44:50.145 INFO:teuthology.orchestra.run.smithi155.stderr: 92.8% -- replaced with /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-mon.smithi155.log.gz 2024-09-19T06:45:23.692 INFO:teuthology.orchestra.run.smithi155.stderr: 93.7% -- replaced with /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-19T06:45:34.391 INFO:teuthology.orchestra.run.smithi155.stderr: 94.0% -- replaced with /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-19T06:45:42.139 INFO:teuthology.orchestra.run.smithi155.stderr: 93.7% -- replaced with /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-19T06:46:20.726 INFO:teuthology.orchestra.run.smithi155.stderr: 94.1% -- replaced with /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-19T06:46:35.063 INFO:teuthology.orchestra.run.smithi053.stderr: 93.9% -- replaced with /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-19T06:47:06.111 INFO:teuthology.orchestra.run.smithi053.stderr: 93.8% -- replaced with /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-19T06:47:09.250 INFO:teuthology.orchestra.run.smithi053.stderr: 94.0% -- replaced with /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-19T06:47:23.518 INFO:teuthology.orchestra.run.smithi053.stderr: 94.2% -- replaced with /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-19T06:52:24.402 INFO:teuthology.orchestra.run.smithi155.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-09-19T06:52:24.403 INFO:teuthology.orchestra.run.smithi155.stderr: 2024-09-19T06:52:24.404 INFO:teuthology.orchestra.run.smithi155.stderr:real 7m58.577s 2024-09-19T06:52:24.404 INFO:teuthology.orchestra.run.smithi155.stderr:user 6m14.499s 2024-09-19T06:52:24.404 INFO:teuthology.orchestra.run.smithi155.stderr:sys 0m16.231s 2024-09-19T06:57:25.746 INFO:teuthology.orchestra.run.smithi053.stderr: 94.6% -- replaced with /var/log/ceph/9f2876d0-764d-11ef-bceb-c7b262605968/ceph-mds.bar.smithi053.nzdkgr.log.gz 2024-09-19T06:57:25.773 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-09-19T06:57:25.773 INFO:teuthology.orchestra.run.smithi053.stderr:real 12m59.940s 2024-09-19T06:57:25.773 INFO:teuthology.orchestra.run.smithi053.stderr:user 9m47.240s 2024-09-19T06:57:25.773 INFO:teuthology.orchestra.run.smithi053.stderr:sys 0m24.062s 2024-09-19T06:57:25.774 INFO:tasks.cephadm:Archiving logs... 2024-09-19T06:57:25.775 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911719/remote/smithi053/log 2024-09-19T06:57:25.776 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T06:58:41.857 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911719/remote/smithi155/log 2024-09-19T06:58:41.858 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T06:59:42.217 INFO:tasks.cephadm:Removing cluster... 2024-09-19T06:59:42.218 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 --force 2024-09-19T06:59:42.951 INFO:teuthology.orchestra.run.smithi053.stdout:Deleting cluster with fsid: 9f2876d0-764d-11ef-bceb-c7b262605968 2024-09-19T06:59:43.619 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9f2876d0-764d-11ef-bceb-c7b262605968 --force 2024-09-19T06:59:44.097 INFO:teuthology.orchestra.run.smithi155.stdout:Deleting cluster with fsid: 9f2876d0-764d-11ef-bceb-c7b262605968 2024-09-19T06:59:44.371 INFO:tasks.cephadm:Removing cephadm ... 2024-09-19T06:59:44.371 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T06:59:44.392 DEBUG:teuthology.orchestra.run.smithi155:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T06:59:44.408 INFO:tasks.cephadm:Teardown complete 2024-09-19T06:59:44.408 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-19T06:59:44.419 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-19T06:59:44.419 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T06:59:44.435 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T06:59:44.553 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-19T06:59:44.562 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-19T06:59:44.563 DEBUG:teuthology.orchestra.run.smithi053:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T06:59:44.565 DEBUG:teuthology.orchestra.run.smithi155:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T06:59:44.580 INFO:teuthology.orchestra.run.smithi155.stderr:bash: line 1: ntpq: command not found 2024-09-19T06:59:44.581 INFO:teuthology.orchestra.run.smithi053.stderr:bash: line 1: ntpq: command not found 2024-09-19T06:59:44.622 INFO:teuthology.orchestra.run.smithi155.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T06:59:44.623 INFO:teuthology.orchestra.run.smithi155.stdout:=============================================================================== 2024-09-19T06:59:44.623 INFO:teuthology.orchestra.run.smithi155.stdout:^* hv01.front.sepia.ceph.com 3 6 377 37 +409us[ +440us] +/- 42ms 2024-09-19T06:59:44.623 INFO:teuthology.orchestra.run.smithi155.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 38 -899us[ -868us] +/- 69ms 2024-09-19T06:59:44.623 INFO:teuthology.orchestra.run.smithi155.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 +230us[ +262us] +/- 66ms 2024-09-19T06:59:44.623 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:59:44.672 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T06:59:44.672 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2024-09-19T06:59:44.672 INFO:teuthology.orchestra.run.smithi053.stdout:^* hv01.front.sepia.ceph.com 3 6 377 36 +428us[ +414us] +/- 42ms 2024-09-19T06:59:44.672 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 39 -890us[ -904us] +/- 69ms 2024-09-19T06:59:44.672 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 40 +221us[ +207us] +/- 66ms 2024-09-19T06:59:44.672 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:59:44.694 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-19T06:59:44.704 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-19T06:59:44.705 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-19T06:59:44.791 DEBUG:teuthology.orchestra.run.smithi053:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T06:59:44.965 DEBUG:teuthology.orchestra.run.smithi155:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T06:59:45.087 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T06:59:45.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T06:59:45.120 DEBUG:teuthology.orchestra.run.smithi155:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T06:59:45.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T06:59:45.148 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-19T06:59:45.159 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-19T06:59:45.196 INFO:teuthology.task.internal:Duration was 3442.394962 seconds 2024-09-19T06:59:45.197 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-19T06:59:45.310 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-19T06:59:45.311 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T06:59:45.314 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T06:59:45.370 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T06:59:45.385 INFO:teuthology.orchestra.run.smithi155.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T06:59:45.794 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-19T06:59:45.794 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi053.front.sepia.ceph.com 2024-09-19T06:59:45.794 DEBUG:teuthology.orchestra.run.smithi053:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T06:59:45.836 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi155.front.sepia.ceph.com 2024-09-19T06:59:45.836 DEBUG:teuthology.orchestra.run.smithi155:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T06:59:45.870 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-19T06:59:45.871 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T06:59:45.881 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T06:59:45.937 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-19T06:59:45.938 DEBUG:teuthology.orchestra.run.smithi053:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T06:59:45.940 DEBUG:teuthology.orchestra.run.smithi155:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T06:59:46.341 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-19T06:59:46.354 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-19T06:59:46.355 DEBUG:teuthology.orchestra.run.smithi053:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T06:59:46.382 DEBUG:teuthology.orchestra.run.smithi155:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T06:59:46.408 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-19T06:59:46.421 DEBUG:teuthology.orchestra.run.smithi053:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T06:59:46.426 DEBUG:teuthology.orchestra.run.smithi155:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T06:59:46.458 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = core 2024-09-19T06:59:46.489 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = core 2024-09-19T06:59:46.513 DEBUG:teuthology.orchestra.run.smithi053:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T06:59:46.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T06:59:46.560 DEBUG:teuthology.orchestra.run.smithi155:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T06:59:46.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T06:59:46.575 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-19T06:59:46.586 INFO:teuthology.task.internal:Transferring archived files... 2024-09-19T06:59:46.588 DEBUG:teuthology.misc:Transferring archived files from smithi053:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911719/remote/smithi053 2024-09-19T06:59:46.588 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T06:59:46.644 DEBUG:teuthology.misc:Transferring archived files from smithi155:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911719/remote/smithi155 2024-09-19T06:59:46.644 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T06:59:46.681 INFO:teuthology.task.internal:Removing archive directory... 2024-09-19T06:59:46.681 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T06:59:46.683 DEBUG:teuthology.orchestra.run.smithi155:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T06:59:46.730 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-19T06:59:46.740 INFO:teuthology.task.internal:Not uploading archives. 2024-09-19T06:59:46.740 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-19T06:59:46.787 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-19T06:59:46.788 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T06:59:46.790 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T06:59:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 06:59 /home/ubuntu/cephtest 2024-09-19T06:59:46.805 INFO:teuthology.orchestra.run.smithi155.stdout: 266029 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 06:59 /home/ubuntu/cephtest 2024-09-19T06:59:46.806 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-19T06:59:46.817 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-19T06:59:46.861 INFO:teuthology.run:Summary data: description: fs/cephadm/multivolume/{0-start 1-mount 2-workload/dbench conf/{client mds mgr mon osd} distro/single-container-host overrides/{ignorelist_health pg_health}} duration: 3442.3949620723724 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-19T06:59:46.861 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T06:59:46.956 INFO:teuthology.run:pass